2024-03-24T11:00:56.160 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-24T11:00:56.160 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/teuthology-2024-03-22_21:08:02-orch-squid-distro-default-smithi/7617877 --name teuthology-2024-03-22_21:08:02-orch-squid-distro-default-smithi --verbose --owner scheduled_teuthology@teuthology --description orch/cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/connectivity} -- /home/teuthworker/archive/teuthology-2024-03-22_21:08:02-orch-squid-distro-default-smithi/7617877/orig.config.yaml 2024-03-24T11:00:56.191 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-24T11:00:56.271 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-03-22_21:08:02-orch-squid-distro-default-smithi/7617877 branch: squid description: orch/cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/connectivity} email: ceph-qa@ceph.com first_in_suite: false job_id: '7617877' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-03-22_21:08:02-orch-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: global: mon election default strategy: 3 mgr: debug mgr: 20 debug ms: 1 mgr/cephadm/use_agent: true mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: db0330b1e4e2470d52b750e251e55a522b4f7d69 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: db0330b1e4e2470d52b750e251e55a522b4f7d69 workunit: branch: squid sha1: f73f1dfd96849300c4322bd7259a3ab37f94c200 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b seed: 8903 sha1: db0330b1e4e2470d52b750e251e55a522b4f7d69 sleep_before_teardown: 0 subset: 44/64 suite: orch suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_f73f1dfd96849300c4322bd7259a3ab37f94c200/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: f73f1dfd96849300c4322bd7259a3ab37f94c200 targets: smithi062.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMKVmzq25YeKw1MgfUfOJywdEoM1CYUWDFwQCCeWPaoDie4Uu9xnlKszomsaVLFYHsRNTz8HZS2SyP3g+hl99BU= smithi139.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJm+byHDUb5PVB6DkMlCOZKTd40GZgfT9nsEGI03HSonCLLa90OYUB+TTEaRVDHiNoRyyHJRDpghgtY2xrRgWpw= tasks: - cephadm: cephadm_branch: v17.2.0 cephadm_git_url: https://github.com/ceph/ceph image: quay.io/ceph/ceph:v17.2.0 - cephadm.shell: mon.a: - ceph config set mgr mgr/cephadm/use_repo_digest false --force - cephadm.shell: env: - sha1 mon.a: - radosgw-admin realm create --rgw-realm=r --default - radosgw-admin zonegroup create --rgw-zonegroup=default --master --default - radosgw-admin zone create --rgw-zonegroup=default --rgw-zone=z --master --default - radosgw-admin period update --rgw-realm=r --commit - ceph orch apply rgw foo --realm r --zone z --placement=2 --port=8000 - ceph orch apply rgw smpl - ceph osd pool create foo - rbd pool init foo - ceph orch apply iscsi foo u p - sleep 120 - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 mon.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph orch upgrade status - ceph health detail - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 - ceph orch ls | grep '^osd ' - cephadm.shell: mon.a: - ceph orch upgrade ls - ceph orch upgrade ls --image quay.io/ceph/ceph --show-all-versions | grep 16.2.0 - ceph orch upgrade ls --image quay.io/ceph/ceph --tags | grep v16.2.2 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-22_21:08:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-24T11:00:56.271 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_f73f1dfd96849300c4322bd7259a3ab37f94c200/qa; will attempt to use it 2024-03-24T11:00:56.272 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_f73f1dfd96849300c4322bd7259a3ab37f94c200/qa/tasks 2024-03-24T11:00:56.272 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-24T11:00:56.273 INFO:teuthology.task.internal:Checking packages... 2024-03-24T11:00:56.297 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'db0330b1e4e2470d52b750e251e55a522b4f7d69' 2024-03-24T11:00:56.297 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-24T11:00:56.297 INFO:teuthology.packaging:ref: None 2024-03-24T11:00:56.297 INFO:teuthology.packaging:tag: None 2024-03-24T11:00:56.297 INFO:teuthology.packaging:branch: squid 2024-03-24T11:00:56.297 INFO:teuthology.packaging:sha1: db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:00:56.298 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-03-24T11:00:56.461 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1764.gf73f1dfd 2024-03-24T11:00:56.462 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-24T11:00:56.469 INFO:teuthology.task.internal:no buildpackages task found 2024-03-24T11:00:56.469 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-24T11:00:56.538 INFO:teuthology.task.internal:Saving configuration 2024-03-24T11:00:56.551 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-24T11:00:56.605 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-24T11:00:56.630 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi062.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-22_21:08:02-orch-squid-distro-default-smithi/7617877', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-24 10:56:03.221993', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMKVmzq25YeKw1MgfUfOJywdEoM1CYUWDFwQCCeWPaoDie4Uu9xnlKszomsaVLFYHsRNTz8HZS2SyP3g+hl99BU='} 2024-03-24T11:00:56.655 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi139.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-22_21:08:02-orch-squid-distro-default-smithi/7617877', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-24 10:56:03.220094', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJm+byHDUb5PVB6DkMlCOZKTd40GZgfT9nsEGI03HSonCLLa90OYUB+TTEaRVDHiNoRyyHJRDpghgtY2xrRgWpw='} 2024-03-24T11:00:56.655 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-24T11:00:56.666 INFO:teuthology.task.internal:roles: ubuntu@smithi062.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a', 'alertmanager.a'] 2024-03-24T11:00:56.666 INFO:teuthology.task.internal:roles: ubuntu@smithi139.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b'] 2024-03-24T11:00:56.666 INFO:teuthology.run_tasks:Running task console_log... 2024-03-24T11:00:56.788 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f66092f50d0>, signals=[15]) 2024-03-24T11:00:56.788 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-24T11:00:56.795 INFO:teuthology.task.internal:Opening connections... 2024-03-24T11:00:56.795 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi062.front.sepia.ceph.com 2024-03-24T11:00:56.797 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-24T11:00:56.905 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi139.front.sepia.ceph.com 2024-03-24T11:00:56.906 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-24T11:00:56.995 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-24T11:00:57.003 DEBUG:teuthology.orchestra.run.smithi062:> uname -m 2024-03-24T11:00:57.021 INFO:teuthology.orchestra.run.smithi062.stdout:x86_64 2024-03-24T11:00:57.022 DEBUG:teuthology.orchestra.run.smithi062:> cat /etc/os-release 2024-03-24T11:00:57.079 INFO:teuthology.orchestra.run.smithi062.stdout:NAME="CentOS Stream" 2024-03-24T11:00:57.079 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION="9" 2024-03-24T11:00:57.080 INFO:teuthology.orchestra.run.smithi062.stdout:ID="centos" 2024-03-24T11:00:57.080 INFO:teuthology.orchestra.run.smithi062.stdout:ID_LIKE="rhel fedora" 2024-03-24T11:00:57.080 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION_ID="9" 2024-03-24T11:00:57.080 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM_ID="platform:el9" 2024-03-24T11:00:57.080 INFO:teuthology.orchestra.run.smithi062.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-24T11:00:57.080 INFO:teuthology.orchestra.run.smithi062.stdout:ANSI_COLOR="0;31" 2024-03-24T11:00:57.080 INFO:teuthology.orchestra.run.smithi062.stdout:LOGO="fedora-logo-icon" 2024-03-24T11:00:57.080 INFO:teuthology.orchestra.run.smithi062.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-24T11:00:57.080 INFO:teuthology.orchestra.run.smithi062.stdout:HOME_URL="https://centos.org/" 2024-03-24T11:00:57.080 INFO:teuthology.orchestra.run.smithi062.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-24T11:00:57.080 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-24T11:00:57.080 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-24T11:00:57.081 INFO:teuthology.lock.ops:Updating smithi062.front.sepia.ceph.com on lock server 2024-03-24T11:00:57.106 DEBUG:teuthology.orchestra.run.smithi139:> uname -m 2024-03-24T11:00:57.123 INFO:teuthology.orchestra.run.smithi139.stdout:x86_64 2024-03-24T11:00:57.123 DEBUG:teuthology.orchestra.run.smithi139:> cat /etc/os-release 2024-03-24T11:00:57.179 INFO:teuthology.orchestra.run.smithi139.stdout:NAME="CentOS Stream" 2024-03-24T11:00:57.179 INFO:teuthology.orchestra.run.smithi139.stdout:VERSION="9" 2024-03-24T11:00:57.180 INFO:teuthology.orchestra.run.smithi139.stdout:ID="centos" 2024-03-24T11:00:57.180 INFO:teuthology.orchestra.run.smithi139.stdout:ID_LIKE="rhel fedora" 2024-03-24T11:00:57.180 INFO:teuthology.orchestra.run.smithi139.stdout:VERSION_ID="9" 2024-03-24T11:00:57.180 INFO:teuthology.orchestra.run.smithi139.stdout:PLATFORM_ID="platform:el9" 2024-03-24T11:00:57.180 INFO:teuthology.orchestra.run.smithi139.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-24T11:00:57.180 INFO:teuthology.orchestra.run.smithi139.stdout:ANSI_COLOR="0;31" 2024-03-24T11:00:57.180 INFO:teuthology.orchestra.run.smithi139.stdout:LOGO="fedora-logo-icon" 2024-03-24T11:00:57.180 INFO:teuthology.orchestra.run.smithi139.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-24T11:00:57.180 INFO:teuthology.orchestra.run.smithi139.stdout:HOME_URL="https://centos.org/" 2024-03-24T11:00:57.180 INFO:teuthology.orchestra.run.smithi139.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-24T11:00:57.180 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-24T11:00:57.180 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-24T11:00:57.181 INFO:teuthology.lock.ops:Updating smithi139.front.sepia.ceph.com on lock server 2024-03-24T11:00:57.206 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-24T11:00:57.215 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-24T11:00:57.293 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-24T11:00:57.293 DEBUG:teuthology.orchestra.run.smithi062:> test '!' -e /home/ubuntu/cephtest 2024-03-24T11:00:57.296 DEBUG:teuthology.orchestra.run.smithi139:> test '!' -e /home/ubuntu/cephtest 2024-03-24T11:00:57.312 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-24T11:00:57.336 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-24T11:00:57.336 DEBUG:teuthology.orchestra.run.smithi062:> test -z $(ls -A /var/lib/ceph) 2024-03-24T11:00:57.353 DEBUG:teuthology.orchestra.run.smithi139:> test -z $(ls -A /var/lib/ceph) 2024-03-24T11:00:57.427 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-24T11:00:57.487 INFO:teuthology.run_tasks:Running task kernel... 2024-03-24T11:00:57.503 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-03-24T11:00:57.503 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-03-24T11:00:57.503 DEBUG:teuthology.orchestra.run.smithi062:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-24T11:00:57.504 DEBUG:teuthology.orchestra.run.smithi139:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-24T11:00:57.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-24T11:00:57.521 DEBUG:teuthology.orchestra.run.smithi139:> uname -r 2024-03-24T11:00:57.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-24T11:00:57.522 DEBUG:teuthology.orchestra.run.smithi062:> uname -r 2024-03-24T11:00:57.578 INFO:teuthology.orchestra.run.smithi139.stdout:5.14.0-361.el9.x86_64 2024-03-24T11:00:57.578 INFO:teuthology.task.kernel:Running kernel on smithi139: 5.14.0-361.el9.x86_64 2024-03-24T11:00:57.578 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum install -y kernel 2024-03-24T11:00:57.580 INFO:teuthology.orchestra.run.smithi062.stdout:5.14.0-361.el9.x86_64 2024-03-24T11:00:57.581 INFO:teuthology.task.kernel:Running kernel on smithi062: 5.14.0-361.el9.x86_64 2024-03-24T11:00:57.581 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum install -y kernel 2024-03-24T11:00:59.573 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS-9 - AppStream 16 MB/s | 6.7 MB 00:00 2024-03-24T11:00:59.792 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-03-24T11:01:01.192 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS-9 - BaseOS 7.8 MB/s | 2.0 MB 00:00 2024-03-24T11:01:01.223 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS-9 - BaseOS 3.7 MB/s | 2.0 MB 00:00 2024-03-24T11:01:01.990 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - BaseOS 18 MB/s | 8.0 MB 00:00 2024-03-24T11:01:02.883 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - BaseOS 5.9 MB/s | 8.0 MB 00:01 2024-03-24T11:01:17.030 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - AppStream 1.3 MB/s | 19 MB 00:13 2024-03-24T11:01:21.281 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - CRB 8.9 MB/s | 6.1 MB 00:00 2024-03-24T11:01:22.965 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - Extras packages 29 kB/s | 16 kB 00:00 2024-03-24T11:01:25.449 INFO:teuthology.orchestra.run.smithi139.stdout:Extra Packages for Enterprise Linux 8.7 MB/s | 21 MB 00:02 2024-03-24T11:01:28.482 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - AppStream 774 kB/s | 19 MB 00:24 2024-03-24T11:01:30.340 INFO:teuthology.orchestra.run.smithi139.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-03-24T11:01:32.430 INFO:teuthology.orchestra.run.smithi139.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-24T11:01:32.431 INFO:teuthology.orchestra.run.smithi139.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-24T11:01:32.535 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-03-24T11:01:32.546 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-03-24T11:01:32.546 INFO:teuthology.orchestra.run.smithi139.stdout: Package Architecture Version Repository Size 2024-03-24T11:01:32.546 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-03-24T11:01:32.547 INFO:teuthology.orchestra.run.smithi139.stdout:Installing: 2024-03-24T11:01:32.547 INFO:teuthology.orchestra.run.smithi139.stdout: kernel x86_64 5.14.0-430.el9 baseos 44 k 2024-03-24T11:01:32.547 INFO:teuthology.orchestra.run.smithi139.stdout:Installing dependencies: 2024-03-24T11:01:32.547 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-core x86_64 5.14.0-430.el9 baseos 15 M 2024-03-24T11:01:32.547 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-modules x86_64 5.14.0-430.el9 baseos 33 M 2024-03-24T11:01:32.547 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-modules-core x86_64 5.14.0-430.el9 baseos 27 M 2024-03-24T11:01:32.547 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-03-24T11:01:32.547 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-03-24T11:01:32.547 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-03-24T11:01:32.547 INFO:teuthology.orchestra.run.smithi139.stdout:Install 4 Packages 2024-03-24T11:01:32.547 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-03-24T11:01:32.548 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 76 M 2024-03-24T11:01:32.548 INFO:teuthology.orchestra.run.smithi139.stdout:Installed size: 123 M 2024-03-24T11:01:32.548 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2024-03-24T11:01:32.640 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.1 MB 00:00 2024-03-24T11:01:32.933 INFO:teuthology.orchestra.run.smithi139.stdout:(1/4): kernel-5.14.0-430.el9.x86_64.rpm 130 kB/s | 44 kB 00:00 2024-03-24T11:01:36.575 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - Extras packages 5.6 kB/s | 16 kB 00:02 2024-03-24T11:01:37.995 INFO:teuthology.orchestra.run.smithi062.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-03-24T11:01:43.167 INFO:teuthology.orchestra.run.smithi062.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-03-24T11:01:45.327 INFO:teuthology.orchestra.run.smithi062.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-24T11:01:45.327 INFO:teuthology.orchestra.run.smithi062.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-24T11:01:45.421 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-03-24T11:01:45.426 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-03-24T11:01:45.426 INFO:teuthology.orchestra.run.smithi062.stdout: Package Architecture Version Repository Size 2024-03-24T11:01:45.426 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-03-24T11:01:45.427 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2024-03-24T11:01:45.427 INFO:teuthology.orchestra.run.smithi062.stdout: kernel x86_64 5.14.0-430.el9 baseos 44 k 2024-03-24T11:01:45.427 INFO:teuthology.orchestra.run.smithi062.stdout:Installing dependencies: 2024-03-24T11:01:45.427 INFO:teuthology.orchestra.run.smithi062.stdout: kernel-core x86_64 5.14.0-430.el9 baseos 15 M 2024-03-24T11:01:45.427 INFO:teuthology.orchestra.run.smithi062.stdout: kernel-modules x86_64 5.14.0-430.el9 baseos 33 M 2024-03-24T11:01:45.427 INFO:teuthology.orchestra.run.smithi062.stdout: kernel-modules-core x86_64 5.14.0-430.el9 baseos 27 M 2024-03-24T11:01:45.427 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-24T11:01:45.427 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2024-03-24T11:01:45.427 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-03-24T11:01:45.427 INFO:teuthology.orchestra.run.smithi062.stdout:Install 4 Packages 2024-03-24T11:01:45.427 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-24T11:01:45.428 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 76 M 2024-03-24T11:01:45.428 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 123 M 2024-03-24T11:01:45.428 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2024-03-24T11:01:45.760 INFO:teuthology.orchestra.run.smithi062.stdout:(1/4): kernel-5.14.0-430.el9.x86_64.rpm 213 kB/s | 44 kB 00:00 2024-03-24T11:01:46.452 INFO:teuthology.orchestra.run.smithi062.stdout:(2/4): kernel-core-5.14.0-430.el9.x86_64.rpm 17 MB/s | 15 MB 00:00 2024-03-24T11:01:46.944 INFO:teuthology.orchestra.run.smithi062.stdout:(3/4): kernel-modules-core-5.14.0-430.el9.x86_6 23 MB/s | 27 MB 00:01 2024-03-24T11:01:47.253 INFO:teuthology.orchestra.run.smithi062.stdout:(4/4): kernel-modules-5.14.0-430.el9.x86_64.rpm 20 MB/s | 33 MB 00:01 2024-03-24T11:01:47.253 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2024-03-24T11:01:47.253 INFO:teuthology.orchestra.run.smithi062.stdout:Total 42 MB/s | 76 MB 00:01 2024-03-24T11:01:47.944 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2024-03-24T11:01:48.315 INFO:teuthology.orchestra.run.smithi139.stdout:(2/4): kernel-core-5.14.0-430.el9.x86_64.rpm 971 kB/s | 15 MB 00:15 2024-03-24T11:01:48.364 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2024-03-24T11:01:48.364 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2024-03-24T11:01:49.397 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2024-03-24T11:01:49.397 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2024-03-24T11:01:51.986 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2024-03-24T11:01:52.614 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : kernel-modules-core-5.14.0-430.el9.x86_64 1/4 2024-03-24T11:01:52.694 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-24T11:01:55.575 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-24T11:01:55.713 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-24T11:02:00.792 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-24T11:02:00.829 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : kernel-5.14.0-430.el9.x86_64 4/4 2024-03-24T11:02:03.580 INFO:teuthology.orchestra.run.smithi139.stdout:(3/4): kernel-modules-core-5.14.0-430.el9.x86_6 918 kB/s | 27 MB 00:30 2024-03-24T11:02:05.932 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-24T11:02:11.870 INFO:teuthology.orchestra.run.smithi139.stdout:(4/4): kernel-modules-5.14.0-430.el9.x86_64.rpm 872 kB/s | 33 MB 00:39 2024-03-24T11:02:11.871 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-03-24T11:02:11.871 INFO:teuthology.orchestra.run.smithi139.stdout:Total 1.9 MB/s | 76 MB 00:39 2024-03-24T11:02:12.538 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-03-24T11:02:12.913 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-03-24T11:02:12.913 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-03-24T11:02:13.880 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-03-24T11:02:13.880 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-03-24T11:02:16.320 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-03-24T11:02:16.963 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : kernel-modules-core-5.14.0-430.el9.x86_64 1/4 2024-03-24T11:02:17.045 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-24T11:02:19.768 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-24T11:02:19.897 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-24T11:02:24.623 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-24T11:02:24.730 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : kernel-5.14.0-430.el9.x86_64 4/4 2024-03-24T11:02:29.438 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-24T11:02:36.732 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 4/4 2024-03-24T11:02:36.733 INFO:teuthology.orchestra.run.smithi062.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-24T11:02:36.733 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-24T11:02:36.778 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 4/4 2024-03-24T11:02:37.186 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: kernel-5.14.0-430.el9.x86_64 4/4 2024-03-24T11:02:37.186 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : kernel-5.14.0-430.el9.x86_64 1/4 2024-03-24T11:02:37.186 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-24T11:02:37.186 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-24T11:02:37.607 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-24T11:02:37.607 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-24T11:02:37.607 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2024-03-24T11:02:37.607 INFO:teuthology.orchestra.run.smithi062.stdout: kernel-5.14.0-430.el9.x86_64 2024-03-24T11:02:37.607 INFO:teuthology.orchestra.run.smithi062.stdout: kernel-core-5.14.0-430.el9.x86_64 2024-03-24T11:02:37.607 INFO:teuthology.orchestra.run.smithi062.stdout: kernel-modules-5.14.0-430.el9.x86_64 2024-03-24T11:02:37.607 INFO:teuthology.orchestra.run.smithi062.stdout: kernel-modules-core-5.14.0-430.el9.x86_64 2024-03-24T11:02:37.607 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-24T11:02:37.607 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-03-24T11:02:37.926 DEBUG:teuthology.orchestra.run.smithi062:> rpm -q kernel | sort -rV | head -n 1 2024-03-24T11:02:38.028 INFO:teuthology.orchestra.run.smithi062.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-24T11:02:38.028 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-24T11:02:38.028 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-430.el9.x86_64 2024-03-24T11:02:38.028 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-24T11:02:38.029 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-24T11:02:38.029 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-03-24T11:02:38.029 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi062.front.sepia.ceph.com, path=None, version=distro) 2024-03-24T11:02:38.029 DEBUG:teuthology.orchestra.run.smithi062:> rpm -q kernel | sort -rV | head -n 1 2024-03-24T11:02:38.111 INFO:teuthology.orchestra.run.smithi062.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-24T11:02:38.111 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-24T11:02:38.111 DEBUG:teuthology.orchestra.run.smithi062:> sudo rpm -qi grub2-tools 2024-03-24T11:02:38.148 INFO:teuthology.orchestra.run.smithi062.stdout:Name : grub2-tools 2024-03-24T11:02:38.148 INFO:teuthology.orchestra.run.smithi062.stdout:Epoch : 1 2024-03-24T11:02:38.148 INFO:teuthology.orchestra.run.smithi062.stdout:Version : 2.06 2024-03-24T11:02:38.149 INFO:teuthology.orchestra.run.smithi062.stdout:Release : 68.el9 2024-03-24T11:02:38.149 INFO:teuthology.orchestra.run.smithi062.stdout:Architecture: x86_64 2024-03-24T11:02:38.149 INFO:teuthology.orchestra.run.smithi062.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-24T11:02:38.149 INFO:teuthology.orchestra.run.smithi062.stdout:Group : Unspecified 2024-03-24T11:02:38.149 INFO:teuthology.orchestra.run.smithi062.stdout:Size : 8264254 2024-03-24T11:02:38.149 INFO:teuthology.orchestra.run.smithi062.stdout:License : GPLv3+ 2024-03-24T11:02:38.149 INFO:teuthology.orchestra.run.smithi062.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-24T11:02:38.149 INFO:teuthology.orchestra.run.smithi062.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-24T11:02:38.150 INFO:teuthology.orchestra.run.smithi062.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-24T11:02:38.150 INFO:teuthology.orchestra.run.smithi062.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-24T11:02:38.150 INFO:teuthology.orchestra.run.smithi062.stdout:Packager : builder@centos.org 2024-03-24T11:02:38.150 INFO:teuthology.orchestra.run.smithi062.stdout:Vendor : CentOS 2024-03-24T11:02:38.150 INFO:teuthology.orchestra.run.smithi062.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-24T11:02:38.150 INFO:teuthology.orchestra.run.smithi062.stdout:Summary : Support tools for GRUB. 2024-03-24T11:02:38.150 INFO:teuthology.orchestra.run.smithi062.stdout:Description : 2024-03-24T11:02:38.150 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-24T11:02:38.150 INFO:teuthology.orchestra.run.smithi062.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-24T11:02:38.150 INFO:teuthology.orchestra.run.smithi062.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-24T11:02:38.150 INFO:teuthology.orchestra.run.smithi062.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-24T11:02:38.150 INFO:teuthology.orchestra.run.smithi062.stdout:hardware devices. 2024-03-24T11:02:38.150 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-24T11:02:38.150 INFO:teuthology.orchestra.run.smithi062.stdout:This subpackage provides tools for support of all platforms. 2024-03-24T11:02:38.151 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-24T11:02:38.152 INFO:teuthology.task.kernel:Updating grub on smithi062 to boot 5.14.0-430.el9.x86_64 2024-03-24T11:02:38.152 DEBUG:teuthology.orchestra.run.smithi062:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-24T11:02:38.761 INFO:teuthology.orchestra.run.smithi062.stderr:Generating grub configuration file ... 2024-03-24T11:02:40.204 INFO:teuthology.orchestra.run.smithi062.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-24T11:02:40.232 INFO:teuthology.orchestra.run.smithi062.stderr:done 2024-03-24T11:02:40.234 DEBUG:teuthology.orchestra.run.smithi062:> mktemp 2024-03-24T11:02:40.250 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/tmp.UjqxnMEUoS 2024-03-24T11:02:40.250 DEBUG:teuthology.orchestra.run.smithi062:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.UjqxnMEUoS 2024-03-24T11:02:40.333 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 0666 /tmp/tmp.UjqxnMEUoS 2024-03-24T11:02:40.477 DEBUG:teuthology.orchestra.remote:smithi062:/tmp/tmp.UjqxnMEUoS is 6KB 2024-03-24T11:02:40.525 DEBUG:teuthology.orchestra.run.smithi062:> rm -fr /tmp/tmp.UjqxnMEUoS 2024-03-24T11:02:40.540 DEBUG:teuthology.orchestra.run.smithi062:> sudo /bin/ls /boot/loader/entries || true 2024-03-24T11:02:40.610 INFO:teuthology.orchestra.run.smithi062.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-24T11:02:40.611 INFO:teuthology.orchestra.run.smithi062.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-24T11:02:40.611 INFO:teuthology.orchestra.run.smithi062.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-24T11:02:40.611 INFO:teuthology.orchestra.run.smithi062.stdout:15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64.conf 2024-03-24T11:02:40.612 DEBUG:teuthology.orchestra.run.smithi062:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64 2024-03-24T11:02:40.776 DEBUG:teuthology.orchestra.run.smithi062:> sudo shutdown -r now 2024-03-24T11:02:58.821 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 4/4 2024-03-24T11:02:58.822 INFO:teuthology.orchestra.run.smithi139.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-24T11:02:58.822 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-03-24T11:02:58.865 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 4/4 2024-03-24T11:02:59.228 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: kernel-5.14.0-430.el9.x86_64 4/4 2024-03-24T11:02:59.228 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : kernel-5.14.0-430.el9.x86_64 1/4 2024-03-24T11:02:59.228 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-24T11:02:59.228 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-24T11:02:59.653 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-24T11:02:59.653 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-03-24T11:02:59.654 INFO:teuthology.orchestra.run.smithi139.stdout:Installed: 2024-03-24T11:02:59.654 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-5.14.0-430.el9.x86_64 2024-03-24T11:02:59.654 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-core-5.14.0-430.el9.x86_64 2024-03-24T11:02:59.654 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-modules-5.14.0-430.el9.x86_64 2024-03-24T11:02:59.654 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-modules-core-5.14.0-430.el9.x86_64 2024-03-24T11:02:59.654 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-03-24T11:02:59.654 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-03-24T11:02:59.991 DEBUG:teuthology.orchestra.run.smithi139:> rpm -q kernel | sort -rV | head -n 1 2024-03-24T11:03:00.087 INFO:teuthology.orchestra.run.smithi139.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-24T11:03:00.087 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-24T11:03:00.087 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-430.el9.x86_64 2024-03-24T11:03:00.087 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-24T11:03:00.087 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-24T11:03:00.087 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-03-24T11:03:00.087 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi139.front.sepia.ceph.com, path=None, version=distro) 2024-03-24T11:03:00.088 DEBUG:teuthology.orchestra.run.smithi139:> rpm -q kernel | sort -rV | head -n 1 2024-03-24T11:03:00.168 INFO:teuthology.orchestra.run.smithi139.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-24T11:03:00.168 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-24T11:03:00.168 DEBUG:teuthology.orchestra.run.smithi139:> sudo rpm -qi grub2-tools 2024-03-24T11:03:00.202 INFO:teuthology.orchestra.run.smithi139.stdout:Name : grub2-tools 2024-03-24T11:03:00.202 INFO:teuthology.orchestra.run.smithi139.stdout:Epoch : 1 2024-03-24T11:03:00.202 INFO:teuthology.orchestra.run.smithi139.stdout:Version : 2.06 2024-03-24T11:03:00.202 INFO:teuthology.orchestra.run.smithi139.stdout:Release : 68.el9 2024-03-24T11:03:00.202 INFO:teuthology.orchestra.run.smithi139.stdout:Architecture: x86_64 2024-03-24T11:03:00.202 INFO:teuthology.orchestra.run.smithi139.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-24T11:03:00.203 INFO:teuthology.orchestra.run.smithi139.stdout:Group : Unspecified 2024-03-24T11:03:00.203 INFO:teuthology.orchestra.run.smithi139.stdout:Size : 8264254 2024-03-24T11:03:00.203 INFO:teuthology.orchestra.run.smithi139.stdout:License : GPLv3+ 2024-03-24T11:03:00.203 INFO:teuthology.orchestra.run.smithi139.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-24T11:03:00.203 INFO:teuthology.orchestra.run.smithi139.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-24T11:03:00.203 INFO:teuthology.orchestra.run.smithi139.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-24T11:03:00.203 INFO:teuthology.orchestra.run.smithi139.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-24T11:03:00.203 INFO:teuthology.orchestra.run.smithi139.stdout:Packager : builder@centos.org 2024-03-24T11:03:00.203 INFO:teuthology.orchestra.run.smithi139.stdout:Vendor : CentOS 2024-03-24T11:03:00.203 INFO:teuthology.orchestra.run.smithi139.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-24T11:03:00.203 INFO:teuthology.orchestra.run.smithi139.stdout:Summary : Support tools for GRUB. 2024-03-24T11:03:00.203 INFO:teuthology.orchestra.run.smithi139.stdout:Description : 2024-03-24T11:03:00.203 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-03-24T11:03:00.203 INFO:teuthology.orchestra.run.smithi139.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-24T11:03:00.203 INFO:teuthology.orchestra.run.smithi139.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-24T11:03:00.204 INFO:teuthology.orchestra.run.smithi139.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-24T11:03:00.204 INFO:teuthology.orchestra.run.smithi139.stdout:hardware devices. 2024-03-24T11:03:00.204 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-03-24T11:03:00.204 INFO:teuthology.orchestra.run.smithi139.stdout:This subpackage provides tools for support of all platforms. 2024-03-24T11:03:00.205 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-24T11:03:00.205 INFO:teuthology.task.kernel:Updating grub on smithi139 to boot 5.14.0-430.el9.x86_64 2024-03-24T11:03:00.205 DEBUG:teuthology.orchestra.run.smithi139:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-24T11:03:00.812 INFO:teuthology.orchestra.run.smithi139.stderr:Generating grub configuration file ... 2024-03-24T11:03:02.128 INFO:teuthology.orchestra.run.smithi139.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-24T11:03:02.175 INFO:teuthology.orchestra.run.smithi139.stderr:done 2024-03-24T11:03:02.177 DEBUG:teuthology.orchestra.run.smithi139:> mktemp 2024-03-24T11:03:02.192 INFO:teuthology.orchestra.run.smithi139.stdout:/tmp/tmp.PS8fEUw80q 2024-03-24T11:03:02.192 DEBUG:teuthology.orchestra.run.smithi139:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.PS8fEUw80q 2024-03-24T11:03:02.261 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 0666 /tmp/tmp.PS8fEUw80q 2024-03-24T11:03:02.397 DEBUG:teuthology.orchestra.remote:smithi139:/tmp/tmp.PS8fEUw80q is 6KB 2024-03-24T11:03:02.451 DEBUG:teuthology.orchestra.run.smithi139:> rm -fr /tmp/tmp.PS8fEUw80q 2024-03-24T11:03:02.467 DEBUG:teuthology.orchestra.run.smithi139:> sudo /bin/ls /boot/loader/entries || true 2024-03-24T11:03:02.539 INFO:teuthology.orchestra.run.smithi139.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-24T11:03:02.540 INFO:teuthology.orchestra.run.smithi139.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-24T11:03:02.540 INFO:teuthology.orchestra.run.smithi139.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-24T11:03:02.540 INFO:teuthology.orchestra.run.smithi139.stdout:15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64.conf 2024-03-24T11:03:02.542 DEBUG:teuthology.orchestra.run.smithi139:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64 2024-03-24T11:03:02.701 DEBUG:teuthology.orchestra.run.smithi139:> sudo shutdown -r now 2024-03-24T11:03:10.788 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-03-24T11:03:10.788 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-03-24T11:03:10.789 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-24T11:03:32.726 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-03-24T11:03:32.727 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-03-24T11:03:32.727 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-24T11:04:10.827 DEBUG:teuthology.orchestra.remote:timed out 2024-03-24T11:04:19.834 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-03-24T11:04:19.835 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-24T11:04:20.267 DEBUG:teuthology.orchestra.run.smithi062:> true 2024-03-24T11:04:20.660 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-03-24T11:04:20.660 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-430.el9.x86_64"... 2024-03-24T11:04:20.661 DEBUG:teuthology.orchestra.run.smithi062:> uname -r 2024-03-24T11:04:20.675 INFO:teuthology.orchestra.run.smithi062.stdout:5.14.0-430.el9.x86_64 2024-03-24T11:04:20.675 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-430.el9.x86_64 vs 5.14.0-430.el9.x86_64 2024-03-24T11:04:20.675 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-24T11:04:20.675 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-24T11:04:21.676 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-03-24T11:04:21.676 DEBUG:teuthology.orchestra.run.smithi062:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-24T11:04:21.741 INFO:teuthology.orchestra.run.smithi062.stdout:ttyS1 2024-03-24T11:04:21.782 DEBUG:teuthology.parallel:result is None 2024-03-24T11:04:32.729 DEBUG:teuthology.orchestra.remote:timed out 2024-03-24T11:04:41.730 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-03-24T11:04:41.732 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-24T11:04:42.164 DEBUG:teuthology.orchestra.run.smithi139:> true 2024-03-24T11:04:42.646 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-03-24T11:04:42.646 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-430.el9.x86_64"... 2024-03-24T11:04:42.646 DEBUG:teuthology.orchestra.run.smithi139:> uname -r 2024-03-24T11:04:42.660 INFO:teuthology.orchestra.run.smithi139.stdout:5.14.0-430.el9.x86_64 2024-03-24T11:04:42.660 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-430.el9.x86_64 vs 5.14.0-430.el9.x86_64 2024-03-24T11:04:42.660 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-24T11:04:42.660 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-24T11:04:43.661 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-03-24T11:04:43.661 DEBUG:teuthology.orchestra.run.smithi139:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-24T11:04:43.740 INFO:teuthology.orchestra.run.smithi139.stdout:ttyS1 2024-03-24T11:04:43.775 DEBUG:teuthology.parallel:result is None 2024-03-24T11:04:43.775 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-24T11:04:43.783 INFO:teuthology.task.internal:Creating test directory... 2024-03-24T11:04:43.783 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-24T11:04:43.786 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-24T11:04:43.801 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-24T11:04:43.820 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-24T11:04:43.854 INFO:teuthology.task.internal:Creating archive directory... 2024-03-24T11:04:43.854 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-24T11:04:43.857 DEBUG:teuthology.orchestra.run.smithi139:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-24T11:04:43.882 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-24T11:04:43.889 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-24T11:04:43.889 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-24T11:04:43.920 DEBUG:teuthology.orchestra.run.smithi139:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-24T11:04:43.950 INFO:teuthology.orchestra.run.smithi139.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-24T11:04:43.959 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-24T11:04:43.963 INFO:teuthology.orchestra.run.smithi139.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-24T11:04:43.974 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-24T11:04:43.975 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-24T11:04:43.982 INFO:teuthology.task.internal:Configuring sudo... 2024-03-24T11:04:43.983 DEBUG:teuthology.orchestra.run.smithi062:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-24T11:04:44.018 DEBUG:teuthology.orchestra.run.smithi139:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-24T11:04:44.046 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-24T11:04:44.055 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-24T11:04:44.056 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-24T11:04:44.088 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-24T11:04:44.103 DEBUG:teuthology.orchestra.run.smithi062:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-24T11:04:44.188 DEBUG:teuthology.orchestra.run.smithi062:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-24T11:04:44.242 DEBUG:teuthology.orchestra.run.smithi062:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-24T11:04:44.324 DEBUG:teuthology.orchestra.run.smithi062:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-24T11:04:44.369 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-24T11:04:44.369 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-24T11:04:44.437 DEBUG:teuthology.orchestra.run.smithi139:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-24T11:04:44.479 DEBUG:teuthology.orchestra.run.smithi139:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-24T11:04:44.525 DEBUG:teuthology.orchestra.run.smithi139:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-24T11:04:44.608 DEBUG:teuthology.orchestra.run.smithi139:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-24T11:04:44.651 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-03-24T11:04:44.651 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-24T11:04:44.718 DEBUG:teuthology.orchestra.run.smithi062:> sudo service rsyslog restart 2024-03-24T11:04:44.721 DEBUG:teuthology.orchestra.run.smithi139:> sudo service rsyslog restart 2024-03-24T11:04:44.769 INFO:teuthology.orchestra.run.smithi062.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-24T11:04:44.820 INFO:teuthology.orchestra.run.smithi139.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-24T11:04:45.263 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-24T11:04:45.272 INFO:teuthology.task.internal:Starting timer... 2024-03-24T11:04:45.272 INFO:teuthology.run_tasks:Running task pcp... 2024-03-24T11:04:45.316 INFO:teuthology.run_tasks:Running task selinux... 2024-03-24T11:04:45.348 DEBUG:teuthology.orchestra.run.smithi062:> sudo service auditd rotate 2024-03-24T11:04:45.411 INFO:teuthology.orchestra.run.smithi062.stdout:Rotating logs: 2024-03-24T11:04:45.413 DEBUG:teuthology.orchestra.run.smithi139:> sudo service auditd rotate 2024-03-24T11:04:45.476 INFO:teuthology.orchestra.run.smithi139.stdout:Rotating logs: 2024-03-24T11:04:45.478 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-24T11:04:45.479 DEBUG:teuthology.orchestra.run.smithi062:> /usr/sbin/getenforce 2024-03-24T11:04:45.498 INFO:teuthology.orchestra.run.smithi062.stdout:Permissive 2024-03-24T11:04:45.498 DEBUG:teuthology.orchestra.run.smithi139:> /usr/sbin/getenforce 2024-03-24T11:04:45.539 INFO:teuthology.orchestra.run.smithi139.stdout:Permissive 2024-03-24T11:04:45.539 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi062.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi139.front.sepia.ceph.com': 'permissive'} 2024-03-24T11:04:45.539 DEBUG:teuthology.orchestra.run.smithi062:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-24T11:04:45.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-24T11:04:45.571 DEBUG:teuthology.orchestra.run.smithi139:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-24T11:04:45.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-24T11:04:45.607 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-24T11:04:45.607 DEBUG:teuthology.orchestra.run.smithi062:> sudo /usr/sbin/setenforce permissive 2024-03-24T11:04:45.650 DEBUG:teuthology.orchestra.run.smithi139:> sudo /usr/sbin/setenforce permissive 2024-03-24T11:04:45.682 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-24T11:04:45.692 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-24T11:04:45.709 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-24T11:04:45.710 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi062.front.sepia.ceph.com,smithi139.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-24T11:09:01.511 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi062.front.sepia.ceph.com'), Remote(name='ubuntu@smithi139.front.sepia.ceph.com')] 2024-03-24T11:09:01.512 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-03-24T11:09:01.513 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-24T11:09:01.607 DEBUG:teuthology.orchestra.run.smithi062:> true 2024-03-24T11:09:01.697 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-03-24T11:09:01.698 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-03-24T11:09:01.698 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-24T11:09:01.790 DEBUG:teuthology.orchestra.run.smithi139:> true 2024-03-24T11:09:01.873 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-03-24T11:09:01.873 INFO:teuthology.run_tasks:Running task clock... 2024-03-24T11:09:01.886 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-24T11:09:01.887 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-24T11:09:01.887 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-24T11:09:01.889 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-24T11:09:01.890 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-24T11:09:01.919 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-24T11:09:01.937 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-24T11:09:01.943 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-24T11:09:01.960 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-24T11:09:01.966 INFO:teuthology.orchestra.run.smithi062.stderr:sudo: ntpd: command not found 2024-03-24T11:09:01.980 INFO:teuthology.orchestra.run.smithi062.stdout:506 Cannot talk to daemon 2024-03-24T11:09:01.989 INFO:teuthology.orchestra.run.smithi139.stderr:sudo: ntpd: command not found 2024-03-24T11:09:01.996 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-24T11:09:02.003 INFO:teuthology.orchestra.run.smithi139.stdout:506 Cannot talk to daemon 2024-03-24T11:09:02.012 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-24T11:09:02.019 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-24T11:09:02.036 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-24T11:09:02.066 INFO:teuthology.orchestra.run.smithi062.stderr:bash: line 1: ntpq: command not found 2024-03-24T11:09:02.070 INFO:teuthology.orchestra.run.smithi062.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-24T11:09:02.070 INFO:teuthology.orchestra.run.smithi062.stdout:=============================================================================== 2024-03-24T11:09:02.070 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T11:09:02.070 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T11:09:02.070 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T11:09:02.070 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T11:09:02.089 INFO:teuthology.orchestra.run.smithi139.stderr:bash: line 1: ntpq: command not found 2024-03-24T11:09:02.093 INFO:teuthology.orchestra.run.smithi139.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-24T11:09:02.093 INFO:teuthology.orchestra.run.smithi139.stdout:=============================================================================== 2024-03-24T11:09:02.093 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T11:09:02.093 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T11:09:02.093 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T11:09:02.094 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T11:09:02.094 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-24T11:09:02.233 INFO:tasks.cephadm:Config: {'cephadm_branch': 'v17.2.0', 'cephadm_git_url': 'https://github.com/ceph/ceph', 'image': 'quay.io/ceph/ceph:v17.2.0', 'conf': {'global': {'mon election default strategy': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1, 'mgr/cephadm/use_agent': True}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': 'db0330b1e4e2470d52b750e251e55a522b4f7d69'} 2024-03-24T11:09:02.233 INFO:tasks.cephadm:Cluster image is quay.io/ceph/ceph:v17.2.0 2024-03-24T11:09:02.234 INFO:tasks.cephadm:Cluster fsid is ebb89568-e9ce-11ee-95cd-87774f69a715 2024-03-24T11:09:02.234 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-24T11:09:02.234 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.62', 'mon.c': '[v2:172.21.15.62:3301,v1:172.21.15.62:6790]', 'mon.b': '172.21.15.139'} 2024-03-24T11:09:02.234 INFO:tasks.cephadm:First mon is mon.a on smithi062 2024-03-24T11:09:02.234 INFO:tasks.cephadm:First mgr is y 2024-03-24T11:09:02.234 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-24T11:09:02.235 DEBUG:teuthology.orchestra.run.smithi062:> sudo hostname $(hostname -s) 2024-03-24T11:09:02.262 DEBUG:teuthology.orchestra.run.smithi139:> sudo hostname $(hostname -s) 2024-03-24T11:09:02.293 INFO:tasks.cephadm:Downloading cephadm (repo https://github.com/ceph/ceph ref v17.2.0)... 2024-03-24T11:09:02.294 DEBUG:teuthology.orchestra.run.smithi062:> curl --silent https://raw.githubusercontent.com/ceph/ceph/v17.2.0/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-24T11:09:02.555 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r--. 1 ubuntu ubuntu 320521 Mar 24 11:09 /home/ubuntu/cephtest/cephadm 2024-03-24T11:09:02.555 DEBUG:teuthology.orchestra.run.smithi139:> curl --silent https://raw.githubusercontent.com/ceph/ceph/v17.2.0/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-24T11:09:02.731 INFO:teuthology.orchestra.run.smithi139.stdout:-rw-r--r--. 1 ubuntu ubuntu 320521 Mar 24 11:09 /home/ubuntu/cephtest/cephadm 2024-03-24T11:09:02.732 DEBUG:teuthology.orchestra.run.smithi062:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-24T11:09:02.748 DEBUG:teuthology.orchestra.run.smithi139:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-24T11:09:02.779 INFO:tasks.cephadm:Pulling image quay.io/ceph/ceph:v17.2.0 on all hosts... 2024-03-24T11:09:02.779 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 pull 2024-03-24T11:09:02.790 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 pull 2024-03-24T11:09:03.659 INFO:teuthology.orchestra.run.smithi139.stderr:Pulling container image quay.io/ceph/ceph:v17.2.0... 2024-03-24T11:09:03.723 INFO:teuthology.orchestra.run.smithi062.stderr:Pulling container image quay.io/ceph/ceph:v17.2.0... 2024-03-24T11:09:20.405 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:09:20.405 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_version": "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)", 2024-03-24T11:09:20.405 INFO:teuthology.orchestra.run.smithi062.stdout: "image_id": "e1d6a67b021eb077ee22bf650f1a9fb1980a2cf5c36bdb9cba9eac6de8f702d9", 2024-03-24T11:09:20.405 INFO:teuthology.orchestra.run.smithi062.stdout: "repo_digests": [ 2024-03-24T11:09:20.405 INFO:teuthology.orchestra.run.smithi062.stdout: "quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a", 2024-03-24T11:09:20.405 INFO:teuthology.orchestra.run.smithi062.stdout: "quay.io/ceph/ceph@sha256:cb4d698cb769b6aba05bf6ef04f41a7fe694160140347576e13bd9348514b667" 2024-03-24T11:09:20.405 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-03-24T11:09:20.405 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:09:24.691 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-03-24T11:09:24.691 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_version": "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)", 2024-03-24T11:09:24.691 INFO:teuthology.orchestra.run.smithi139.stdout: "image_id": "e1d6a67b021eb077ee22bf650f1a9fb1980a2cf5c36bdb9cba9eac6de8f702d9", 2024-03-24T11:09:24.691 INFO:teuthology.orchestra.run.smithi139.stdout: "repo_digests": [ 2024-03-24T11:09:24.692 INFO:teuthology.orchestra.run.smithi139.stdout: "quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a", 2024-03-24T11:09:24.692 INFO:teuthology.orchestra.run.smithi139.stdout: "quay.io/ceph/ceph@sha256:cb4d698cb769b6aba05bf6ef04f41a7fe694160140347576e13bd9348514b667" 2024-03-24T11:09:24.692 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-03-24T11:09:24.692 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-03-24T11:09:24.710 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /etc/ceph 2024-03-24T11:09:24.744 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /etc/ceph 2024-03-24T11:09:24.775 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 777 /etc/ceph 2024-03-24T11:09:24.815 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 777 /etc/ceph 2024-03-24T11:09:24.842 INFO:tasks.cephadm:Writing seed config... 2024-03-24T11:09:24.843 INFO:tasks.cephadm: override: [global] mon election default strategy = 3 2024-03-24T11:09:24.843 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-24T11:09:24.843 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-24T11:09:24.843 INFO:tasks.cephadm: override: [mgr] mgr/cephadm/use_agent = True 2024-03-24T11:09:24.843 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-24T11:09:24.843 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-24T11:09:24.843 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-24T11:09:24.843 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-24T11:09:24.843 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-24T11:09:24.844 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-24T11:09:24.844 DEBUG:teuthology.orchestra.run.smithi062:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-24T11:09:24.872 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = ebb89568-e9ce-11ee-95cd-87774f69a715 mon election default strategy = 3 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 mgr/cephadm/use_agent = True [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-03-24T11:09:24.873 DEBUG:teuthology.orchestra.run.smithi062:mon.a> sudo journalctl -f -n 0 -u ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mon.a.service 2024-03-24T11:09:24.916 DEBUG:teuthology.orchestra.run.smithi062:mgr.y> sudo journalctl -f -n 0 -u ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mgr.y.service 2024-03-24T11:09:24.959 INFO:tasks.cephadm:Bootstrapping... 2024-03-24T11:09:24.959 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 -v bootstrap --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.62 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-24T11:09:25.165 INFO:teuthology.orchestra.run.smithi062.stderr:-------------------------------------------------------------------------------- 2024-03-24T11:09:25.166 INFO:teuthology.orchestra.run.smithi062.stderr:cephadm ['--image', 'quay.io/ceph/ceph:v17.2.0', '-v', 'bootstrap', '--fsid', 'ebb89568-e9ce-11ee-95cd-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.62', '--skip-admin-label'] 2024-03-24T11:09:25.189 INFO:teuthology.orchestra.run.smithi062.stderr:/bin/podman: 4.9.3-dev 2024-03-24T11:09:25.190 INFO:teuthology.orchestra.run.smithi062.stderr:Verifying podman|docker is present... 2024-03-24T11:09:25.212 INFO:teuthology.orchestra.run.smithi062.stderr:/bin/podman: 4.9.3-dev 2024-03-24T11:09:25.214 INFO:teuthology.orchestra.run.smithi062.stderr:Verifying lvm2 is present... 2024-03-24T11:09:25.214 INFO:teuthology.orchestra.run.smithi062.stderr:Verifying time synchronization is in place... 2024-03-24T11:09:25.221 INFO:teuthology.orchestra.run.smithi062.stderr:systemctl: Failed to get unit file state for chrony.service: No such file or directory 2024-03-24T11:09:25.228 INFO:teuthology.orchestra.run.smithi062.stderr:systemctl: inactive 2024-03-24T11:09:25.234 INFO:teuthology.orchestra.run.smithi062.stderr:systemctl: enabled 2024-03-24T11:09:25.241 INFO:teuthology.orchestra.run.smithi062.stderr:systemctl: active 2024-03-24T11:09:25.242 INFO:teuthology.orchestra.run.smithi062.stderr:Unit chronyd.service is enabled and running 2024-03-24T11:09:25.242 INFO:teuthology.orchestra.run.smithi062.stderr:Repeating the final host check... 2024-03-24T11:09:25.263 INFO:teuthology.orchestra.run.smithi062.stderr:/bin/podman: 4.9.3-dev 2024-03-24T11:09:25.265 INFO:teuthology.orchestra.run.smithi062.stderr:podman (/bin/podman) version 4.9.3 is present 2024-03-24T11:09:25.265 INFO:teuthology.orchestra.run.smithi062.stderr:systemctl is present 2024-03-24T11:09:25.265 INFO:teuthology.orchestra.run.smithi062.stderr:lvcreate is present 2024-03-24T11:09:25.271 INFO:teuthology.orchestra.run.smithi062.stderr:systemctl: Failed to get unit file state for chrony.service: No such file or directory 2024-03-24T11:09:25.278 INFO:teuthology.orchestra.run.smithi062.stderr:systemctl: inactive 2024-03-24T11:09:25.285 INFO:teuthology.orchestra.run.smithi062.stderr:systemctl: enabled 2024-03-24T11:09:25.291 INFO:teuthology.orchestra.run.smithi062.stderr:systemctl: active 2024-03-24T11:09:25.292 INFO:teuthology.orchestra.run.smithi062.stderr:Unit chronyd.service is enabled and running 2024-03-24T11:09:25.292 INFO:teuthology.orchestra.run.smithi062.stderr:Host looks OK 2024-03-24T11:09:25.293 INFO:teuthology.orchestra.run.smithi062.stderr:Cluster fsid: ebb89568-e9ce-11ee-95cd-87774f69a715 2024-03-24T11:09:25.293 INFO:teuthology.orchestra.run.smithi062.stderr:Acquiring lock 140287080357360 on /run/cephadm/ebb89568-e9ce-11ee-95cd-87774f69a715.lock 2024-03-24T11:09:25.293 INFO:teuthology.orchestra.run.smithi062.stderr:Lock 140287080357360 acquired on /run/cephadm/ebb89568-e9ce-11ee-95cd-87774f69a715.lock 2024-03-24T11:09:25.293 INFO:teuthology.orchestra.run.smithi062.stderr:Verifying IP 172.21.15.62 port 3300 ... 2024-03-24T11:09:25.293 INFO:teuthology.orchestra.run.smithi062.stderr:Verifying IP 172.21.15.62 port 6789 ... 2024-03-24T11:09:25.293 INFO:teuthology.orchestra.run.smithi062.stderr:Base mon IP is 172.21.15.62, final addrv is [v2:172.21.15.62:3300,v1:172.21.15.62:6789] 2024-03-24T11:09:25.296 INFO:teuthology.orchestra.run.smithi062.stderr:/sbin/ip: default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.62 metric 100 2024-03-24T11:09:25.296 INFO:teuthology.orchestra.run.smithi062.stderr:/sbin/ip: 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.62 metric 100 2024-03-24T11:09:25.299 INFO:teuthology.orchestra.run.smithi062.stderr:/sbin/ip: ::1 dev lo proto kernel metric 256 pref medium 2024-03-24T11:09:25.299 INFO:teuthology.orchestra.run.smithi062.stderr:/sbin/ip: fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-03-24T11:09:25.299 INFO:teuthology.orchestra.run.smithi062.stderr:/sbin/ip: default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1774sec hoplimit 64 pref medium 2024-03-24T11:09:25.301 INFO:teuthology.orchestra.run.smithi062.stderr:/sbin/ip: 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-24T11:09:25.301 INFO:teuthology.orchestra.run.smithi062.stderr:/sbin/ip: inet6 ::1/128 scope host 2024-03-24T11:09:25.301 INFO:teuthology.orchestra.run.smithi062.stderr:/sbin/ip: valid_lft forever preferred_lft forever 2024-03-24T11:09:25.301 INFO:teuthology.orchestra.run.smithi062.stderr:/sbin/ip: 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-03-24T11:09:25.301 INFO:teuthology.orchestra.run.smithi062.stderr:/sbin/ip: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2024-03-24T11:09:25.301 INFO:teuthology.orchestra.run.smithi062.stderr:/sbin/ip: valid_lft forever preferred_lft forever 2024-03-24T11:09:25.302 INFO:teuthology.orchestra.run.smithi062.stderr:Mon IP `172.21.15.62` is in CIDR network `172.21.0.0/20` 2024-03-24T11:09:25.302 INFO:teuthology.orchestra.run.smithi062.stderr:- internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-24T11:09:25.303 INFO:teuthology.orchestra.run.smithi062.stderr:Pulling container image quay.io/ceph/ceph:v17.2.0... 2024-03-24T11:09:25.407 INFO:teuthology.orchestra.run.smithi062.stderr:/bin/podman: Trying to pull quay.io/ceph/ceph:v17.2.0... 2024-03-24T11:09:25.848 INFO:teuthology.orchestra.run.smithi062.stderr:/bin/podman: Getting image source signatures 2024-03-24T11:09:25.848 INFO:teuthology.orchestra.run.smithi062.stderr:/bin/podman: Copying blob sha256:33ca8fff7868c4dc0c11e09bca97c720eb9cfbab7221216754367dd8de70388a 2024-03-24T11:09:25.848 INFO:teuthology.orchestra.run.smithi062.stderr:/bin/podman: Copying blob sha256:599d07cb321ff0a3c82224e1138fc685793fa69b93ed5780415751a5f7e4b8c2 2024-03-24T11:09:25.848 INFO:teuthology.orchestra.run.smithi062.stderr:/bin/podman: Copying blob sha256:c32ab78b488d0c72f64eded765c0cf6b5bf2c75dab66cb62a9d367fa6ec42513 2024-03-24T11:09:25.848 INFO:teuthology.orchestra.run.smithi062.stderr:/bin/podman: Copying blob sha256:a70843738bb77e1ab9c1f85969ebdfa55f178e746be081d1cb4f94011f69eb7c 2024-03-24T11:09:25.848 INFO:teuthology.orchestra.run.smithi062.stderr:/bin/podman: Copying blob sha256:89b4a75bc2d8500f15463747507c9623df43886c134463e7f0527e70900e7a7b 2024-03-24T11:09:25.848 INFO:teuthology.orchestra.run.smithi062.stderr:/bin/podman: Copying config sha256:e1d6a67b021eb077ee22bf650f1a9fb1980a2cf5c36bdb9cba9eac6de8f702d9 2024-03-24T11:09:25.966 INFO:teuthology.orchestra.run.smithi062.stderr:/bin/podman: Writing manifest to image destination 2024-03-24T11:09:26.102 INFO:teuthology.orchestra.run.smithi062.stderr:/bin/podman: e1d6a67b021eb077ee22bf650f1a9fb1980a2cf5c36bdb9cba9eac6de8f702d9 2024-03-24T11:09:26.454 INFO:teuthology.orchestra.run.smithi062.stderr:ceph: ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable) 2024-03-24T11:09:27.010 INFO:teuthology.orchestra.run.smithi062.stderr:Ceph version: ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable) 2024-03-24T11:09:27.010 INFO:teuthology.orchestra.run.smithi062.stderr:Extracting ceph user uid/gid from container image... 2024-03-24T11:09:27.417 INFO:teuthology.orchestra.run.smithi062.stderr:stat: 167 167 2024-03-24T11:09:27.890 INFO:teuthology.orchestra.run.smithi062.stderr:Creating initial keys... 2024-03-24T11:09:28.265 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph-authtool: AQDoCQBmC6C6CxAA1rftoWnhZelNS8FAFdqX9g== 2024-03-24T11:09:29.170 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph-authtool: AQDpCQBm5a9CBRAAsvJi4gcVhjWzLLROSQPcAA== 2024-03-24T11:09:30.025 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph-authtool: AQDpCQBm/L2fOhAAxKrrF/APXdp44UC7iQkG0g== 2024-03-24T11:09:30.508 INFO:teuthology.orchestra.run.smithi062.stderr:Creating initial monmap... 2024-03-24T11:09:30.898 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/monmaptool: /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-24T11:09:30.898 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/monmaptool: setting min_mon_release = octopus 2024-03-24T11:09:30.898 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/monmaptool: /usr/bin/monmaptool: set fsid to ebb89568-e9ce-11ee-95cd-87774f69a715 2024-03-24T11:09:30.898 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/monmaptool: /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-24T11:09:31.388 INFO:teuthology.orchestra.run.smithi062.stderr:monmaptool for a [v2:172.21.15.62:3300,v1:172.21.15.62:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-24T11:09:31.388 INFO:teuthology.orchestra.run.smithi062.stderr:setting min_mon_release = octopus 2024-03-24T11:09:31.388 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/monmaptool: set fsid to ebb89568-e9ce-11ee-95cd-87774f69a715 2024-03-24T11:09:31.388 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-24T11:09:31.388 INFO:teuthology.orchestra.run.smithi062.stderr: 2024-03-24T11:09:31.389 INFO:teuthology.orchestra.run.smithi062.stderr:Creating mon... 2024-03-24T11:09:32.310 INFO:teuthology.orchestra.run.smithi062.stderr:create mon.a on 2024-03-24T11:09:32.534 INFO:teuthology.orchestra.run.smithi062.stderr:systemctl: Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-24T11:09:32.683 INFO:teuthology.orchestra.run.smithi062.stderr:systemctl: Created symlink /etc/systemd/system/multi-user.target.wants/ceph-ebb89568-e9ce-11ee-95cd-87774f69a715.target → /etc/systemd/system/ceph-ebb89568-e9ce-11ee-95cd-87774f69a715.target. 2024-03-24T11:09:32.683 INFO:teuthology.orchestra.run.smithi062.stderr:systemctl: Created symlink /etc/systemd/system/ceph.target.wants/ceph-ebb89568-e9ce-11ee-95cd-87774f69a715.target → /etc/systemd/system/ceph-ebb89568-e9ce-11ee-95cd-87774f69a715.target. 2024-03-24T11:09:32.979 INFO:teuthology.orchestra.run.smithi062.stderr:systemctl: Failed to reset failed state of unit ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mon.a.service: Unit ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mon.a.service not loaded. 2024-03-24T11:09:32.986 INFO:teuthology.orchestra.run.smithi062.stderr:systemctl: Created symlink /etc/systemd/system/ceph-ebb89568-e9ce-11ee-95cd-87774f69a715.target.wants/ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mon.a.service → /etc/systemd/system/ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@.service. 2024-03-24T11:09:33.431 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:33 smithi062 podman[24441]: 2024-03-24 11:09:33.286150038 +0000 UTC m=+0.018489384 image pull e1d6a67b021eb077ee22bf650f1a9fb1980a2cf5c36bdb9cba9eac6de8f702d9 quay.io/ceph/ceph:v17.2.0 2024-03-24T11:09:33.623 INFO:teuthology.orchestra.run.smithi062.stderr:systemctl: disabled 2024-03-24T11:09:33.629 INFO:teuthology.orchestra.run.smithi062.stderr:systemctl: inactive 2024-03-24T11:09:33.630 INFO:teuthology.orchestra.run.smithi062.stderr:firewalld.service is not enabled 2024-03-24T11:09:33.630 INFO:teuthology.orchestra.run.smithi062.stderr:Not possible to enable service . firewalld.service is not available 2024-03-24T11:09:33.630 INFO:teuthology.orchestra.run.smithi062.stderr:Waiting for mon to start... 2024-03-24T11:09:33.630 INFO:teuthology.orchestra.run.smithi062.stderr:Waiting for mon... 2024-03-24T11:09:33.726 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:33 smithi062 podman[24441]: 2024-03-24 11:09:33.511671282 +0000 UTC m=+0.244010611 container init 6227f0b3c6e0a1acd181ea32ebc38d237dd270e55a006ce216a3e78b1ab9768b (image=quay.io/ceph/ceph:v17.2.0, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-a, architecture=x86_64, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, ceph=True, GIT_REPO=https://github.com/ceph/ceph-container.git, vendor=Red Hat, Inc., version=8, name=centos-stream, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, io.k8s.display-name=CentOS Stream 8, GIT_BRANCH=HEAD, com.redhat.component=centos-stream-container, release=754, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.openshift.tags=base centos centos-stream, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, CEPH_POINT_RELEASE=-17.2.0, RELEASE=HEAD, build-date=2022-05-03T08:36:31.336870, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, maintainer=Guillaume Abrioux , io.openshift.expose-services=, vcs-type=git, distribution-scope=public, io.buildah.version=1.19.8, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, GIT_CLEAN=True, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb) 2024-03-24T11:09:33.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:33 smithi062 podman[24441]: 2024-03-24 11:09:33.514779169 +0000 UTC m=+0.247118496 container start 6227f0b3c6e0a1acd181ea32ebc38d237dd270e55a006ce216a3e78b1ab9768b (image=quay.io/ceph/ceph:v17.2.0, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-a, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, vcs-type=git, RELEASE=HEAD, com.redhat.component=centos-stream-container, name=centos-stream, distribution-scope=public, architecture=x86_64, GIT_BRANCH=HEAD, io.openshift.expose-services=, io.buildah.version=1.19.8, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, build-date=2022-05-03T08:36:31.336870, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., version=8, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, GIT_REPO=https://github.com/ceph/ceph-container.git, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, ceph=True, io.k8s.display-name=CentOS Stream 8, GIT_CLEAN=True, vendor=Red Hat, Inc., com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, CEPH_POINT_RELEASE=-17.2.0, maintainer=Guillaume Abrioux , release=754, io.openshift.tags=base centos centos-stream) 2024-03-24T11:09:33.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:33 smithi062 ceph-mon[24459]: mkfs ebb89568-e9ce-11ee-95cd-87774f69a715 2024-03-24T11:09:33.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:33 smithi062 ceph-mon[24459]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-24T11:09:33.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:33 smithi062 bash[24441]: 6227f0b3c6e0a1acd181ea32ebc38d237dd270e55a006ce216a3e78b1ab9768b 2024-03-24T11:09:33.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:33 smithi062 systemd[1]: Started Ceph mon.a for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:09:34.133 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: cluster: 2024-03-24T11:09:34.133 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: id: ebb89568-e9ce-11ee-95cd-87774f69a715 2024-03-24T11:09:34.133 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: health: HEALTH_OK 2024-03-24T11:09:34.133 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: 2024-03-24T11:09:34.134 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: services: 2024-03-24T11:09:34.134 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: mon: 1 daemons, quorum a (age 0.557446s) 2024-03-24T11:09:34.134 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: mgr: no daemons active 2024-03-24T11:09:34.134 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: osd: 0 osds: 0 up, 0 in 2024-03-24T11:09:34.134 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: 2024-03-24T11:09:34.134 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: data: 2024-03-24T11:09:34.134 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: pools: 0 pools, 0 pgs 2024-03-24T11:09:34.134 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: objects: 0 objects, 0 B 2024-03-24T11:09:34.134 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: usage: 0 B used, 0 B / 0 B avail 2024-03-24T11:09:34.134 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: pgs: 2024-03-24T11:09:34.134 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: 2024-03-24T11:09:34.630 INFO:teuthology.orchestra.run.smithi062.stderr:mon is available 2024-03-24T11:09:34.630 INFO:teuthology.orchestra.run.smithi062.stderr:Assimilating anything we can from ceph.conf... 2024-03-24T11:09:34.726 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:34 smithi062 ceph-mon[24459]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-24T11:09:34.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:34 smithi062 ceph-mon[24459]: monmap e1: 1 mons at {a=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0]} 2024-03-24T11:09:34.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:34 smithi062 ceph-mon[24459]: fsmap 2024-03-24T11:09:34.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:34 smithi062 ceph-mon[24459]: osdmap e1: 0 total, 0 up, 0 in 2024-03-24T11:09:34.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:34 smithi062 ceph-mon[24459]: mgrmap e1: no daemons active 2024-03-24T11:09:34.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:34 smithi062 ceph-mon[24459]: from='client.? 172.21.15.62:0/1615043739' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-24T11:09:35.090 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: 2024-03-24T11:09:35.090 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: [global] 2024-03-24T11:09:35.090 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: fsid = ebb89568-e9ce-11ee-95cd-87774f69a715 2024-03-24T11:09:35.091 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: mon_host = [v2:172.21.15.62:3300,v1:172.21.15.62:6789] 2024-03-24T11:09:35.091 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: mon_osd_allow_pg_remap = true 2024-03-24T11:09:35.091 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: mon_osd_allow_primary_affinity = true 2024-03-24T11:09:35.091 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: mon_warn_on_no_sortbitwise = false 2024-03-24T11:09:35.091 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: osd_crush_chooseleaf_type = 0 2024-03-24T11:09:35.091 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: 2024-03-24T11:09:35.091 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: [mgr] 2024-03-24T11:09:35.091 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: mgr/cephadm/use_agent = True 2024-03-24T11:09:35.092 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: mgr/telemetry/nag = false 2024-03-24T11:09:35.092 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: 2024-03-24T11:09:35.092 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: [osd] 2024-03-24T11:09:35.092 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: osd_map_max_advance = 10 2024-03-24T11:09:35.092 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: osd_sloppy_crc = true 2024-03-24T11:09:35.665 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:35 smithi062 ceph-mon[24459]: from='client.? 172.21.15.62:0/305622454' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-24T11:09:35.665 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:35 smithi062 ceph-mon[24459]: from='client.? 172.21.15.62:0/305622454' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-24T11:09:35.667 INFO:teuthology.orchestra.run.smithi062.stderr:Generating new minimal ceph.conf... 2024-03-24T11:09:36.607 INFO:teuthology.orchestra.run.smithi062.stderr:Restarting the monitor... 2024-03-24T11:09:36.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:36 smithi062 ceph-mon[24459]: from='client.? 172.21.15.62:0/1829776043' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:09:36.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:36 smithi062 systemd[1]: Stopping Ceph mon.a for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:09:36.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:36 smithi062 bash[24678]: Error: no container with name or ID "ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon.a" found: no such container 2024-03-24T11:09:36.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:36 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-a[24455]: 2024-03-24T11:09:36.747+0000 7f64faf92700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T11:09:36.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:36 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-a[24455]: 2024-03-24T11:09:36.747+0000 7f64faf92700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-24T11:09:36.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:36 smithi062 podman[24688]: 2024-03-24 11:09:36.820606013 +0000 UTC m=+0.166250513 container died 6227f0b3c6e0a1acd181ea32ebc38d237dd270e55a006ce216a3e78b1ab9768b (image=quay.io/ceph/ceph:v17.2.0, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-a, GIT_BRANCH=HEAD, io.k8s.display-name=CentOS Stream 8, GIT_CLEAN=True, release=754, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., architecture=x86_64, distribution-scope=public, vendor=Red Hat, Inc., vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, RELEASE=HEAD, io.buildah.version=1.19.8, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, CEPH_POINT_RELEASE=-17.2.0, version=8, build-date=2022-05-03T08:36:31.336870, GIT_REPO=https://github.com/ceph/ceph-container.git, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, ceph=True, maintainer=Guillaume Abrioux , io.openshift.tags=base centos centos-stream, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, com.redhat.component=centos-stream-container, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, io.openshift.expose-services=, name=centos-stream, vcs-type=git, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream) 2024-03-24T11:09:37.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 podman[24688]: 2024-03-24 11:09:37.148950648 +0000 UTC m=+0.494595147 container cleanup 6227f0b3c6e0a1acd181ea32ebc38d237dd270e55a006ce216a3e78b1ab9768b (image=quay.io/ceph/ceph:v17.2.0, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-a, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, version=8, GIT_CLEAN=True, ceph=True, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, CEPH_POINT_RELEASE=-17.2.0, RELEASE=HEAD, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, io.k8s.display-name=CentOS Stream 8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, distribution-scope=public, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, io.openshift.tags=base centos centos-stream, vendor=Red Hat, Inc., io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., GIT_REPO=https://github.com/ceph/ceph-container.git, io.openshift.expose-services=, architecture=x86_64, build-date=2022-05-03T08:36:31.336870, vcs-type=git, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, maintainer=Guillaume Abrioux , com.redhat.component=centos-stream-container, io.buildah.version=1.19.8, GIT_BRANCH=HEAD, release=754, name=centos-stream, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac) 2024-03-24T11:09:37.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 bash[24688]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-a 2024-03-24T11:09:37.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 podman[24702]: 2024-03-24 11:09:37.333203078 +0000 UTC m=+0.517432236 container remove 6227f0b3c6e0a1acd181ea32ebc38d237dd270e55a006ce216a3e78b1ab9768b (image=quay.io/ceph/ceph:v17.2.0, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-a, RELEASE=HEAD, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, GIT_REPO=https://github.com/ceph/ceph-container.git, release=754, GIT_BRANCH=HEAD, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, io.buildah.version=1.19.8, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, maintainer=Guillaume Abrioux , com.redhat.component=centos-stream-container, vendor=Red Hat, Inc., ceph=True, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, name=centos-stream, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, io.k8s.display-name=CentOS Stream 8, CEPH_POINT_RELEASE=-17.2.0, architecture=x86_64, io.openshift.tags=base centos centos-stream, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, vcs-type=git, io.openshift.expose-services=, build-date=2022-05-03T08:36:31.336870, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., distribution-scope=public, version=8, GIT_CLEAN=True) 2024-03-24T11:09:37.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 bash[24714]: Error: no container with name or ID "ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon.a" found: no such container 2024-03-24T11:09:37.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-24T11:09:37.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 systemd[1]: Stopped Ceph mon.a for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:09:37.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 systemd[1]: Starting Ceph mon.a for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:09:37.805 INFO:teuthology.orchestra.run.smithi062.stderr:Setting mon public_network to 172.21.0.0/20 2024-03-24T11:09:37.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 podman[24780]: 2024-03-24 11:09:37.58391718 +0000 UTC m=+0.102671271 container create 712f01aa91804babf25132e8257ba078ccc0905c25c6c191c3daa9e15b989df4 (image=quay.io/ceph/ceph:v17.2.0, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-a, maintainer=Guillaume Abrioux , vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, com.redhat.component=centos-stream-container, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, vendor=Red Hat, Inc., CEPH_POINT_RELEASE=-17.2.0, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, io.k8s.display-name=CentOS Stream 8, name=centos-stream, GIT_BRANCH=HEAD, ceph=True, RELEASE=HEAD, distribution-scope=public, build-date=2022-05-03T08:36:31.336870, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, architecture=x86_64, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, io.buildah.version=1.19.8, GIT_CLEAN=True, release=754, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, vcs-type=git, GIT_REPO=https://github.com/ceph/ceph-container.git, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.openshift.expose-services=, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, version=8, io.openshift.tags=base centos centos-stream) 2024-03-24T11:09:37.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 podman[24780]: 2024-03-24 11:09:37.499844416 +0000 UTC m=+0.018598528 image pull e1d6a67b021eb077ee22bf650f1a9fb1980a2cf5c36bdb9cba9eac6de8f702d9 quay.io/ceph/ceph:v17.2.0 2024-03-24T11:09:37.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 podman[24780]: 2024-03-24 11:09:37.736529134 +0000 UTC m=+0.255283239 container init 712f01aa91804babf25132e8257ba078ccc0905c25c6c191c3daa9e15b989df4 (image=quay.io/ceph/ceph:v17.2.0, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-a, build-date=2022-05-03T08:36:31.336870, GIT_CLEAN=True, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, RELEASE=HEAD, vendor=Red Hat, Inc., com.redhat.component=centos-stream-container, CEPH_POINT_RELEASE=-17.2.0, release=754, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, GIT_REPO=https://github.com/ceph/ceph-container.git, io.openshift.expose-services=, name=centos-stream, vcs-type=git, architecture=x86_64, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., maintainer=Guillaume Abrioux , vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, io.k8s.display-name=CentOS Stream 8, io.buildah.version=1.19.8, version=8, io.openshift.tags=base centos centos-stream, GIT_BRANCH=HEAD, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, ceph=True, distribution-scope=public, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754) 2024-03-24T11:09:37.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 podman[24780]: 2024-03-24 11:09:37.739325071 +0000 UTC m=+0.258079166 container start 712f01aa91804babf25132e8257ba078ccc0905c25c6c191c3daa9e15b989df4 (image=quay.io/ceph/ceph:v17.2.0, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-a, vcs-type=git, maintainer=Guillaume Abrioux , io.buildah.version=1.19.8, vendor=Red Hat, Inc., CEPH_POINT_RELEASE=-17.2.0, io.k8s.display-name=CentOS Stream 8, version=8, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, io.openshift.expose-services=, distribution-scope=public, build-date=2022-05-03T08:36:31.336870, ceph=True, GIT_CLEAN=True, architecture=x86_64, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, com.redhat.component=centos-stream-container, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_BRANCH=HEAD, io.openshift.tags=base centos centos-stream, release=754, RELEASE=HEAD, name=centos-stream) 2024-03-24T11:09:37.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: set uid:gid to 167:167 (ceph:ceph) 2024-03-24T11:09:37.840 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable), process ceph-mon, pid 2 2024-03-24T11:09:37.840 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: pidfile_write: ignore empty --pid-file 2024-03-24T11:09:37.840 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: load: jerasure load: lrc 2024-03-24T11:09:37.840 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: RocksDB version: 6.15.5 2024-03-24T11:09:37.840 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-03-24T11:09:37.840 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Compile date Apr 18 2022 2024-03-24T11:09:37.840 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: DB SUMMARY 2024-03-24T11:09:37.840 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: DB Session ID: 7STLBQ1LTIQ38NTYFQF5 2024-03-24T11:09:37.840 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: CURRENT file: CURRENT 2024-03-24T11:09:37.840 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: IDENTITY file: IDENTITY 2024-03-24T11:09:37.840 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2024-03-24T11:09:37.840 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-24T11:09:37.840 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000010.log size: 82629 ; 2024-03-24T11:09:37.841 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.error_if_exists: 0 2024-03-24T11:09:37.841 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.create_if_missing: 0 2024-03-24T11:09:37.841 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.paranoid_checks: 1 2024-03-24T11:09:37.841 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-24T11:09:37.841 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.env: 0x5604b68c6860 2024-03-24T11:09:37.841 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.fs: Posix File System 2024-03-24T11:09:37.841 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.info_log: 0x5604b8ca7dc0 2024-03-24T11:09:37.841 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.max_file_opening_threads: 16 2024-03-24T11:09:37.841 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.statistics: (nil) 2024-03-24T11:09:37.841 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.use_fsync: 0 2024-03-24T11:09:37.841 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.max_log_file_size: 0 2024-03-24T11:09:37.841 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-24T11:09:37.842 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-24T11:09:37.842 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.keep_log_file_num: 1000 2024-03-24T11:09:37.842 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.recycle_log_file_num: 0 2024-03-24T11:09:37.842 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.allow_fallocate: 1 2024-03-24T11:09:37.842 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.allow_mmap_reads: 0 2024-03-24T11:09:37.842 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.allow_mmap_writes: 0 2024-03-24T11:09:37.842 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.use_direct_reads: 0 2024-03-24T11:09:37.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-24T11:09:37.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.create_missing_column_families: 0 2024-03-24T11:09:37.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.db_log_dir: 2024-03-24T11:09:37.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-a/store.db 2024-03-24T11:09:37.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-24T11:09:37.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-24T11:09:37.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-24T11:09:37.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-24T11:09:37.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-24T11:09:37.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-24T11:09:37.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.advise_random_on_open: 1 2024-03-24T11:09:37.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.db_write_buffer_size: 0 2024-03-24T11:09:37.843 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.write_buffer_manager: 0x5604b8d982a0 2024-03-24T11:09:37.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-24T11:09:37.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-03-24T11:09:37.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-24T11:09:37.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-24T11:09:37.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.rate_limiter: (nil) 2024-03-24T11:09:37.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-24T11:09:37.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.wal_recovery_mode: 2 2024-03-24T11:09:37.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.enable_thread_tracking: 0 2024-03-24T11:09:37.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.enable_pipelined_write: 0 2024-03-24T11:09:37.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.unordered_write: 0 2024-03-24T11:09:37.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-24T11:09:37.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-24T11:09:37.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-24T11:09:37.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-24T11:09:37.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.row_cache: None 2024-03-24T11:09:37.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.wal_filter: None 2024-03-24T11:09:37.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-24T11:09:37.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.allow_ingest_behind: 0 2024-03-24T11:09:37.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.preserve_deletes: 0 2024-03-24T11:09:37.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.two_write_queues: 0 2024-03-24T11:09:37.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.manual_wal_flush: 0 2024-03-24T11:09:37.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.atomic_flush: 0 2024-03-24T11:09:37.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-24T11:09:37.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-24T11:09:37.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-24T11:09:37.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.log_readahead_size: 0 2024-03-24T11:09:37.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-24T11:09:37.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.best_efforts_recovery: 0 2024-03-24T11:09:37.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-24T11:09:37.846 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-24T11:09:37.846 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.allow_data_in_errors: 0 2024-03-24T11:09:37.846 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.db_host_id: __hostname__ 2024-03-24T11:09:37.846 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.max_background_jobs: 2 2024-03-24T11:09:37.846 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.max_background_compactions: -1 2024-03-24T11:09:37.846 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.max_subcompactions: 1 2024-03-24T11:09:37.846 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-24T11:09:37.846 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-24T11:09:37.846 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-24T11:09:37.846 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.max_total_wal_size: 0 2024-03-24T11:09:37.846 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-24T11:09:37.846 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-24T11:09:37.846 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-24T11:09:37.846 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-24T11:09:37.847 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.max_open_files: -1 2024-03-24T11:09:37.847 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.bytes_per_sync: 0 2024-03-24T11:09:37.847 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-24T11:09:37.847 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-24T11:09:37.847 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.compaction_readahead_size: 0 2024-03-24T11:09:37.847 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.max_background_flushes: -1 2024-03-24T11:09:37.847 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Compression algorithms supported: 2024-03-24T11:09:37.847 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-24T11:09:37.847 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: kZSTD supported: 0 2024-03-24T11:09:37.847 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: kXpressCompression supported: 0 2024-03-24T11:09:37.847 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: kLZ4HCCompression supported: 1 2024-03-24T11:09:37.847 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: kLZ4Compression supported: 1 2024-03-24T11:09:37.847 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: kBZip2Compression supported: 0 2024-03-24T11:09:37.847 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: kZlibCompression supported: 1 2024-03-24T11:09:37.847 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: kSnappyCompression supported: 1 2024-03-24T11:09:37.848 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-24T11:09:37.848 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: [db/version_set.cc:4725] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 2024-03-24T11:09:37.848 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: [db/column_family.cc:597] --------------- Options for column family [default]: 2024-03-24T11:09:37.848 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-24T11:09:37.848 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.merge_operator: 2024-03-24T11:09:37.848 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.compaction_filter: None 2024-03-24T11:09:37.848 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.compaction_filter_factory: None 2024-03-24T11:09:37.848 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.sst_partitioner_factory: None 2024-03-24T11:09:37.848 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-24T11:09:37.848 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-24T11:09:37.848 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5604b8c73d00) 2024-03-24T11:09:37.848 INFO:journalctl@ceph.mon.a.smithi062.stdout: cache_index_and_filter_blocks: 1 2024-03-24T11:09:37.848 INFO:journalctl@ceph.mon.a.smithi062.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-24T11:09:37.848 INFO:journalctl@ceph.mon.a.smithi062.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-24T11:09:37.849 INFO:journalctl@ceph.mon.a.smithi062.stdout: pin_top_level_index_and_filter: 1 2024-03-24T11:09:37.849 INFO:journalctl@ceph.mon.a.smithi062.stdout: index_type: 0 2024-03-24T11:09:37.849 INFO:journalctl@ceph.mon.a.smithi062.stdout: data_block_index_type: 0 2024-03-24T11:09:37.849 INFO:journalctl@ceph.mon.a.smithi062.stdout: index_shortening: 1 2024-03-24T11:09:37.849 INFO:journalctl@ceph.mon.a.smithi062.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-24T11:09:37.849 INFO:journalctl@ceph.mon.a.smithi062.stdout: hash_index_allow_collision: 1 2024-03-24T11:09:37.849 INFO:journalctl@ceph.mon.a.smithi062.stdout: checksum: 1 2024-03-24T11:09:37.849 INFO:journalctl@ceph.mon.a.smithi062.stdout: no_block_cache: 0 2024-03-24T11:09:37.849 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_cache: 0x5604b8cde170 2024-03-24T11:09:37.849 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_cache_name: BinnedLRUCache 2024-03-24T11:09:37.849 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_cache_options: 2024-03-24T11:09:37.849 INFO:journalctl@ceph.mon.a.smithi062.stdout: capacity : 536870912 2024-03-24T11:09:37.849 INFO:journalctl@ceph.mon.a.smithi062.stdout: num_shard_bits : 4 2024-03-24T11:09:37.849 INFO:journalctl@ceph.mon.a.smithi062.stdout: strict_capacity_limit : 0 2024-03-24T11:09:37.850 INFO:journalctl@ceph.mon.a.smithi062.stdout: high_pri_pool_ratio: 0.000 2024-03-24T11:09:37.850 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_cache_compressed: (nil) 2024-03-24T11:09:37.850 INFO:journalctl@ceph.mon.a.smithi062.stdout: persistent_cache: (nil) 2024-03-24T11:09:37.850 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_size: 4096 2024-03-24T11:09:37.850 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_size_deviation: 10 2024-03-24T11:09:37.850 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_restart_interval: 16 2024-03-24T11:09:37.850 INFO:journalctl@ceph.mon.a.smithi062.stdout: index_block_restart_interval: 1 2024-03-24T11:09:37.850 INFO:journalctl@ceph.mon.a.smithi062.stdout: metadata_block_size: 4096 2024-03-24T11:09:37.850 INFO:journalctl@ceph.mon.a.smithi062.stdout: partition_filters: 0 2024-03-24T11:09:37.850 INFO:journalctl@ceph.mon.a.smithi062.stdout: use_delta_encoding: 1 2024-03-24T11:09:37.850 INFO:journalctl@ceph.mon.a.smithi062.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-03-24T11:09:37.850 INFO:journalctl@ceph.mon.a.smithi062.stdout: whole_key_filtering: 1 2024-03-24T11:09:37.850 INFO:journalctl@ceph.mon.a.smithi062.stdout: verify_compression: 0 2024-03-24T11:09:37.850 INFO:journalctl@ceph.mon.a.smithi062.stdout: read_amp_bytes_per_bit: 0 2024-03-24T11:09:37.850 INFO:journalctl@ceph.mon.a.smithi062.stdout: format_version: 4 2024-03-24T11:09:37.851 INFO:journalctl@ceph.mon.a.smithi062.stdout: enable_index_compression: 1 2024-03-24T11:09:37.851 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_align: 0 2024-03-24T11:09:37.851 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.write_buffer_size: 33554432 2024-03-24T11:09:37.851 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.max_write_buffer_number: 2 2024-03-24T11:09:37.851 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.compression: NoCompression 2024-03-24T11:09:37.851 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.bottommost_compression: Disabled 2024-03-24T11:09:37.851 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.prefix_extractor: nullptr 2024-03-24T11:09:37.851 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-24T11:09:37.851 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.num_levels: 7 2024-03-24T11:09:37.852 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-24T11:09:37.852 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-24T11:09:37.852 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-24T11:09:37.852 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-24T11:09:37.852 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-24T11:09:37.853 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-24T11:09:37.853 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-24T11:09:37.853 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-24T11:09:37.853 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-24T11:09:37.853 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-24T11:09:37.853 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-24T11:09:37.853 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.compression_opts.level: 32767 2024-03-24T11:09:37.853 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.compression_opts.strategy: 0 2024-03-24T11:09:37.853 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-24T11:09:37.853 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-24T11:09:37.853 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-24T11:09:37.853 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.compression_opts.enabled: false 2024-03-24T11:09:37.853 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-24T11:09:37.853 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-24T11:09:37.853 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-24T11:09:37.854 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.target_file_size_base: 67108864 2024-03-24T11:09:37.854 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-24T11:09:37.854 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-24T11:09:37.854 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-24T11:09:37.854 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-24T11:09:37.854 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-24T11:09:37.854 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-24T11:09:37.854 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-24T11:09:37.854 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-24T11:09:37.854 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-24T11:09:37.854 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-24T11:09:37.854 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-24T11:09:37.854 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-24T11:09:37.855 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-24T11:09:37.855 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.arena_block_size: 4194304 2024-03-24T11:09:37.855 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-24T11:09:37.855 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-24T11:09:37.855 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-03-24T11:09:37.855 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.disable_auto_compactions: 0 2024-03-24T11:09:37.855 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-24T11:09:37.855 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-24T11:09:37.855 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-24T11:09:37.855 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-24T11:09:37.855 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-24T11:09:37.855 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-24T11:09:37.855 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-24T11:09:37.855 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-24T11:09:37.855 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-24T11:09:37.856 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-24T11:09:37.856 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.table_properties_collectors: 2024-03-24T11:09:37.856 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.inplace_update_support: 0 2024-03-24T11:09:37.856 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-24T11:09:37.856 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-24T11:09:37.856 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-24T11:09:37.856 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-24T11:09:37.856 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.bloom_locality: 0 2024-03-24T11:09:37.856 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.max_successive_merges: 0 2024-03-24T11:09:37.856 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-24T11:09:37.856 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.paranoid_file_checks: 0 2024-03-24T11:09:37.856 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.force_consistency_checks: 1 2024-03-24T11:09:37.856 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.report_bg_io_stats: 0 2024-03-24T11:09:37.856 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.ttl: 2592000 2024-03-24T11:09:37.856 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-24T11:09:37.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.enable_blob_files: false 2024-03-24T11:09:37.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.min_blob_size: 0 2024-03-24T11:09:37.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.blob_file_size: 268435456 2024-03-24T11:09:37.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-24T11:09:37.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-24T11:09:37.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-24T11:09:37.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: [db/version_set.cc:4773] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-24T11:09:37.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: [db/version_set.cc:4782] Column family [default] (ID 0), log number is 5 2024-03-24T11:09:37.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: [db/version_set.cc:4083] Creating manifest 13 2024-03-24T11:09:37.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711278577766342, "job": 1, "event": "recovery_started", "wal_files": [10]} 2024-03-24T11:09:37.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: [db/db_impl/db_impl_open.cc:847] Recovering log #10 mode 2 2024-03-24T11:09:37.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: [table/block_based/filter_policy.cc:996] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-03-24T11:09:37.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711278577767036, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 79179, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 77442, "index_size": 225, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 10027, "raw_average_key_size": 47, "raw_value_size": 71751, "raw_average_value_size": 340, "num_data_blocks": 10, "num_entries": 211, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1711278577, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "9fb791f4-da1a-43d7-9aee-0e7b374ab2d2", "db_session_id": "7STLBQ1LTIQ38NTYFQF5"}} 2024-03-24T11:09:37.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: [db/version_set.cc:4083] Creating manifest 15 2024-03-24T11:09:37.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711278577767395, "job": 1, "event": "recovery_finished"} 2024-03-24T11:09:37.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: [file/delete_scheduler.cc:73] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-24T11:09:37.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: [db/db_impl/db_impl_open.cc:1701] SstFileManager instance 0x5604b8cc4a80 2024-03-24T11:09:37.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: DB pointer 0x5604b8cd4000 2024-03-24T11:09:37.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: [db/db_impl/db_impl.cc:902] ------- DUMPING STATS ------- 2024-03-24T11:09:37.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-03-24T11:09:37.858 INFO:journalctl@ceph.mon.a.smithi062.stdout: ** DB Stats ** 2024-03-24T11:09:37.858 INFO:journalctl@ceph.mon.a.smithi062.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-24T11:09:37.858 INFO:journalctl@ceph.mon.a.smithi062.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-24T11:09:37.858 INFO:journalctl@ceph.mon.a.smithi062.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-24T11:09:37.858 INFO:journalctl@ceph.mon.a.smithi062.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-24T11:09:37.858 INFO:journalctl@ceph.mon.a.smithi062.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-24T11:09:37.858 INFO:journalctl@ceph.mon.a.smithi062.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-03-24T11:09:37.858 INFO:journalctl@ceph.mon.a.smithi062.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-24T11:09:37.859 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2024-03-24T11:09:37.859 INFO:journalctl@ceph.mon.a.smithi062.stdout: ** Compaction Stats [default] ** 2024-03-24T11:09:37.859 INFO:journalctl@ceph.mon.a.smithi062.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-24T11:09:37.859 INFO:journalctl@ceph.mon.a.smithi062.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-24T11:09:37.859 INFO:journalctl@ceph.mon.a.smithi062.stdout: L0 2/0 79.08 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 159.6 0.00 0.00 1 0.000 0 0 2024-03-24T11:09:37.859 INFO:journalctl@ceph.mon.a.smithi062.stdout: Sum 2/0 79.08 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 159.6 0.00 0.00 1 0.000 0 0 2024-03-24T11:09:37.859 INFO:journalctl@ceph.mon.a.smithi062.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 159.6 0.00 0.00 1 0.000 0 0 2024-03-24T11:09:37.859 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2024-03-24T11:09:37.859 INFO:journalctl@ceph.mon.a.smithi062.stdout: ** Compaction Stats [default] ** 2024-03-24T11:09:37.859 INFO:journalctl@ceph.mon.a.smithi062.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-24T11:09:37.859 INFO:journalctl@ceph.mon.a.smithi062.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-24T11:09:37.859 INFO:journalctl@ceph.mon.a.smithi062.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 159.6 0.00 0.00 1 0.000 0 0 2024-03-24T11:09:37.859 INFO:journalctl@ceph.mon.a.smithi062.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-24T11:09:37.859 INFO:journalctl@ceph.mon.a.smithi062.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-24T11:09:37.860 INFO:journalctl@ceph.mon.a.smithi062.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-24T11:09:37.860 INFO:journalctl@ceph.mon.a.smithi062.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-24T11:09:37.860 INFO:journalctl@ceph.mon.a.smithi062.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-24T11:09:37.860 INFO:journalctl@ceph.mon.a.smithi062.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-24T11:09:37.860 INFO:journalctl@ceph.mon.a.smithi062.stdout: Cumulative compaction: 0.00 GB write, 25.72 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-24T11:09:37.860 INFO:journalctl@ceph.mon.a.smithi062.stdout: Interval compaction: 0.00 GB write, 25.72 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-24T11:09:37.860 INFO:journalctl@ceph.mon.a.smithi062.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-24T11:09:37.860 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2024-03-24T11:09:37.860 INFO:journalctl@ceph.mon.a.smithi062.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-24T11:09:37.860 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2024-03-24T11:09:37.860 INFO:journalctl@ceph.mon.a.smithi062.stdout: ** Compaction Stats [default] ** 2024-03-24T11:09:37.860 INFO:journalctl@ceph.mon.a.smithi062.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-24T11:09:37.860 INFO:journalctl@ceph.mon.a.smithi062.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-24T11:09:37.860 INFO:journalctl@ceph.mon.a.smithi062.stdout: L0 2/0 79.08 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 159.6 0.00 0.00 1 0.000 0 0 2024-03-24T11:09:37.860 INFO:journalctl@ceph.mon.a.smithi062.stdout: Sum 2/0 79.08 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 159.6 0.00 0.00 1 0.000 0 0 2024-03-24T11:09:37.861 INFO:journalctl@ceph.mon.a.smithi062.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-03-24T11:09:37.861 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2024-03-24T11:09:37.861 INFO:journalctl@ceph.mon.a.smithi062.stdout: ** Compaction Stats [default] ** 2024-03-24T11:09:37.861 INFO:journalctl@ceph.mon.a.smithi062.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-24T11:09:37.861 INFO:journalctl@ceph.mon.a.smithi062.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-24T11:09:37.861 INFO:journalctl@ceph.mon.a.smithi062.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 159.6 0.00 0.00 1 0.000 0 0 2024-03-24T11:09:37.861 INFO:journalctl@ceph.mon.a.smithi062.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-24T11:09:37.861 INFO:journalctl@ceph.mon.a.smithi062.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-24T11:09:37.861 INFO:journalctl@ceph.mon.a.smithi062.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-24T11:09:37.861 INFO:journalctl@ceph.mon.a.smithi062.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-24T11:09:37.861 INFO:journalctl@ceph.mon.a.smithi062.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-24T11:09:37.861 INFO:journalctl@ceph.mon.a.smithi062.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-24T11:09:37.861 INFO:journalctl@ceph.mon.a.smithi062.stdout: Cumulative compaction: 0.00 GB write, 25.42 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-24T11:09:37.861 INFO:journalctl@ceph.mon.a.smithi062.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-24T11:09:37.861 INFO:journalctl@ceph.mon.a.smithi062.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-24T11:09:37.862 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2024-03-24T11:09:37.862 INFO:journalctl@ceph.mon.a.smithi062.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-24T11:09:37.862 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: starting mon.a rank 0 at public addrs [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] at bind addrs [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid ebb89568-e9ce-11ee-95cd-87774f69a715 2024-03-24T11:09:37.862 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: mon.a@-1(???) e1 preinit fsid ebb89568-e9ce-11ee-95cd-87774f69a715 2024-03-24T11:09:37.862 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: mon.a@-1(???).mds e1 new map 2024-03-24T11:09:37.862 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: mon.a@-1(???).mds e1 print_map 2024-03-24T11:09:37.862 INFO:journalctl@ceph.mon.a.smithi062.stdout: e1 2024-03-24T11:09:37.862 INFO:journalctl@ceph.mon.a.smithi062.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T11:09:37.862 INFO:journalctl@ceph.mon.a.smithi062.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T11:09:37.862 INFO:journalctl@ceph.mon.a.smithi062.stdout: legacy client fscid: -1 2024-03-24T11:09:37.862 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2024-03-24T11:09:37.862 INFO:journalctl@ceph.mon.a.smithi062.stdout: No filesystems configured 2024-03-24T11:09:37.862 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-24T11:09:37.862 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-24T11:09:37.862 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-24T11:09:37.863 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-24T11:09:37.863 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-24T11:09:37.863 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: expand_channel_meta expand map: {default=false} 2024-03-24T11:09:37.863 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: expand_channel_meta from 'false' to 'false' 2024-03-24T11:09:37.863 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: expand_channel_meta expanded map: {default=false} 2024-03-24T11:09:37.863 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: expand_channel_meta expand map: {default=info} 2024-03-24T11:09:37.863 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: expand_channel_meta from 'info' to 'info' 2024-03-24T11:09:37.863 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: expand_channel_meta expanded map: {default=info} 2024-03-24T11:09:37.863 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: expand_channel_meta expand map: {default=daemon} 2024-03-24T11:09:37.863 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: expand_channel_meta from 'daemon' to 'daemon' 2024-03-24T11:09:37.863 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: expand_channel_meta expanded map: {default=daemon} 2024-03-24T11:09:37.863 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: expand_channel_meta expand map: {default=debug} 2024-03-24T11:09:37.863 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: expand_channel_meta from 'debug' to 'debug' 2024-03-24T11:09:37.863 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: expand_channel_meta expanded map: {default=debug} 2024-03-24T11:09:37.863 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 bash[24780]: 712f01aa91804babf25132e8257ba078ccc0905c25c6c191c3daa9e15b989df4 2024-03-24T11:09:37.864 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 systemd[1]: Started Ceph mon.a for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:09:37.864 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-24T11:09:37.864 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: monmap e1: 1 mons at {a=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0]} 2024-03-24T11:09:37.864 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: fsmap 2024-03-24T11:09:37.864 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: osdmap e1: 0 total, 0 up, 0 in 2024-03-24T11:09:37.864 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:37 smithi062 ceph-mon[24799]: mgrmap e1: no daemons active 2024-03-24T11:09:38.820 INFO:teuthology.orchestra.run.smithi062.stderr:Wrote config to /etc/ceph/ceph.conf 2024-03-24T11:09:38.820 INFO:teuthology.orchestra.run.smithi062.stderr:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-24T11:09:38.820 INFO:teuthology.orchestra.run.smithi062.stderr:Creating mgr... 2024-03-24T11:09:38.820 INFO:teuthology.orchestra.run.smithi062.stderr:Verifying port 9283 ... 2024-03-24T11:09:38.989 INFO:teuthology.orchestra.run.smithi062.stderr:systemctl: Failed to reset failed state of unit ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mgr.y.service: Unit ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mgr.y.service not loaded. 2024-03-24T11:09:38.996 INFO:teuthology.orchestra.run.smithi062.stderr:systemctl: Created symlink /etc/systemd/system/ceph-ebb89568-e9ce-11ee-95cd-87774f69a715.target.wants/ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mgr.y.service → /etc/systemd/system/ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@.service. 2024-03-24T11:09:39.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:39 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/3940898757' entity='client.admin' 2024-03-24T11:09:39.477 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:09:39 smithi062 podman[25027]: 2024-03-24 11:09:39.385767031 +0000 UTC m=+0.125372128 container create 28a8eee2781708aeffac2cb97b1de29cee6b717a1a4a92db247661393e42e0c9 (image=quay.io/ceph/ceph:v17.2.0, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y, io.openshift.tags=base centos centos-stream, version=8, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, io.k8s.display-name=CentOS Stream 8, RELEASE=HEAD, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., distribution-scope=public, release=754, io.buildah.version=1.19.8, architecture=x86_64, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, io.openshift.expose-services=, ceph=True, GIT_BRANCH=HEAD, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, GIT_CLEAN=True, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, com.redhat.component=centos-stream-container, vcs-type=git, CEPH_POINT_RELEASE=-17.2.0, name=centos-stream, vendor=Red Hat, Inc., maintainer=Guillaume Abrioux , build-date=2022-05-03T08:36:31.336870) 2024-03-24T11:09:39.700 INFO:teuthology.orchestra.run.smithi062.stderr:systemctl: disabled 2024-03-24T11:09:39.707 INFO:teuthology.orchestra.run.smithi062.stderr:systemctl: inactive 2024-03-24T11:09:39.708 INFO:teuthology.orchestra.run.smithi062.stderr:firewalld.service is not enabled 2024-03-24T11:09:39.708 INFO:teuthology.orchestra.run.smithi062.stderr:Not possible to enable service . firewalld.service is not available 2024-03-24T11:09:39.716 INFO:teuthology.orchestra.run.smithi062.stderr:systemctl: disabled 2024-03-24T11:09:39.723 INFO:teuthology.orchestra.run.smithi062.stderr:systemctl: inactive 2024-03-24T11:09:39.724 INFO:teuthology.orchestra.run.smithi062.stderr:firewalld.service is not enabled 2024-03-24T11:09:39.724 INFO:teuthology.orchestra.run.smithi062.stderr:Not possible to open ports <[9283]>. firewalld.service is not available 2024-03-24T11:09:39.724 INFO:teuthology.orchestra.run.smithi062.stderr:Waiting for mgr to start... 2024-03-24T11:09:39.724 INFO:teuthology.orchestra.run.smithi062.stderr:Waiting for mgr... 2024-03-24T11:09:39.851 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:09:39 smithi062 podman[25027]: 2024-03-24 11:09:39.587770093 +0000 UTC m=+0.327375149 container init 28a8eee2781708aeffac2cb97b1de29cee6b717a1a4a92db247661393e42e0c9 (image=quay.io/ceph/ceph:v17.2.0, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, vcs-type=git, GIT_BRANCH=HEAD, io.k8s.display-name=CentOS Stream 8, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, io.openshift.tags=base centos centos-stream, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git, io.openshift.expose-services=, io.buildah.version=1.19.8, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, GIT_CLEAN=True, com.redhat.component=centos-stream-container, ceph=True, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, version=8, vendor=Red Hat, Inc., CEPH_POINT_RELEASE=-17.2.0, distribution-scope=public, build-date=2022-05-03T08:36:31.336870, name=centos-stream, release=754, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, architecture=x86_64, RELEASE=HEAD) 2024-03-24T11:09:39.851 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:09:39 smithi062 podman[25027]: 2024-03-24 11:09:39.590388854 +0000 UTC m=+0.329993910 container start 28a8eee2781708aeffac2cb97b1de29cee6b717a1a4a92db247661393e42e0c9 (image=quay.io/ceph/ceph:v17.2.0, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y, architecture=x86_64, GIT_CLEAN=True, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, distribution-scope=public, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, GIT_REPO=https://github.com/ceph/ceph-container.git, vcs-type=git, io.openshift.expose-services=, release=754, io.k8s.display-name=CentOS Stream 8, maintainer=Guillaume Abrioux , build-date=2022-05-03T08:36:31.336870, name=centos-stream, GIT_BRANCH=HEAD, RELEASE=HEAD, CEPH_POINT_RELEASE=-17.2.0, version=8, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, ceph=True, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, io.openshift.tags=base centos centos-stream, com.redhat.component=centos-stream-container, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, io.buildah.version=1.19.8, vendor=Red Hat, Inc., io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream) 2024-03-24T11:09:39.851 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:09:39 smithi062 bash[25027]: 28a8eee2781708aeffac2cb97b1de29cee6b717a1a4a92db247661393e42e0c9 2024-03-24T11:09:39.851 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:09:39 smithi062 systemd[1]: Started Ceph mgr.y for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:09:40.107 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:09:40 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:09:40.096+0000 7fbcbd9ce000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-24T11:09:40.287 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: 2024-03-24T11:09:40.287 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: { 2024-03-24T11:09:40.287 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "fsid": "ebb89568-e9ce-11ee-95cd-87774f69a715", 2024-03-24T11:09:40.287 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "health": { 2024-03-24T11:09:40.287 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "status": "HEALTH_OK", 2024-03-24T11:09:40.287 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "checks": {}, 2024-03-24T11:09:40.287 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "mutes": [] 2024-03-24T11:09:40.287 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: }, 2024-03-24T11:09:40.288 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "election_epoch": 5, 2024-03-24T11:09:40.288 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "quorum": [ 2024-03-24T11:09:40.288 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: 0 2024-03-24T11:09:40.288 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: ], 2024-03-24T11:09:40.288 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "quorum_names": [ 2024-03-24T11:09:40.288 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "a" 2024-03-24T11:09:40.288 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: ], 2024-03-24T11:09:40.288 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "quorum_age": 2, 2024-03-24T11:09:40.288 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "monmap": { 2024-03-24T11:09:40.288 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "epoch": 1, 2024-03-24T11:09:40.288 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "min_mon_release_name": "quincy", 2024-03-24T11:09:40.288 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "num_mons": 1 2024-03-24T11:09:40.288 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: }, 2024-03-24T11:09:40.288 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "osdmap": { 2024-03-24T11:09:40.289 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "epoch": 1, 2024-03-24T11:09:40.289 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "num_osds": 0, 2024-03-24T11:09:40.289 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "num_up_osds": 0, 2024-03-24T11:09:40.289 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "osd_up_since": 0, 2024-03-24T11:09:40.289 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "num_in_osds": 0, 2024-03-24T11:09:40.292 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "osd_in_since": 0, 2024-03-24T11:09:40.292 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "num_remapped_pgs": 0 2024-03-24T11:09:40.292 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: }, 2024-03-24T11:09:40.292 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "pgmap": { 2024-03-24T11:09:40.292 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "pgs_by_state": [], 2024-03-24T11:09:40.293 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "num_pgs": 0, 2024-03-24T11:09:40.293 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "num_pools": 0, 2024-03-24T11:09:40.293 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "num_objects": 0, 2024-03-24T11:09:40.293 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "data_bytes": 0, 2024-03-24T11:09:40.293 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "bytes_used": 0, 2024-03-24T11:09:40.293 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "bytes_avail": 0, 2024-03-24T11:09:40.293 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "bytes_total": 0 2024-03-24T11:09:40.293 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: }, 2024-03-24T11:09:40.293 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "fsmap": { 2024-03-24T11:09:40.293 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "epoch": 1, 2024-03-24T11:09:40.293 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "by_rank": [], 2024-03-24T11:09:40.293 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "up:standby": 0 2024-03-24T11:09:40.293 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: }, 2024-03-24T11:09:40.293 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "mgrmap": { 2024-03-24T11:09:40.294 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "available": false, 2024-03-24T11:09:40.294 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "num_standbys": 0, 2024-03-24T11:09:40.294 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "modules": [ 2024-03-24T11:09:40.294 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "iostat", 2024-03-24T11:09:40.294 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "nfs", 2024-03-24T11:09:40.294 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "restful" 2024-03-24T11:09:40.294 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: ], 2024-03-24T11:09:40.294 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "services": {} 2024-03-24T11:09:40.294 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: }, 2024-03-24T11:09:40.294 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "servicemap": { 2024-03-24T11:09:40.294 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "epoch": 1, 2024-03-24T11:09:40.294 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "modified": "2024-03-24T11:09:33.575913+0000", 2024-03-24T11:09:40.294 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "services": {} 2024-03-24T11:09:40.294 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: }, 2024-03-24T11:09:40.295 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "progress_events": {} 2024-03-24T11:09:40.295 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: } 2024-03-24T11:09:40.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:40 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/95751062' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-24T11:09:40.476 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:09:40 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:09:40.190+0000 7fbcbd9ce000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-24T11:09:40.780 INFO:teuthology.orchestra.run.smithi062.stderr:mgr not available, waiting (1/15)... 2024-03-24T11:09:41.136 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:09:40 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:09:40.811+0000 7fbcbd9ce000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-24T11:09:41.136 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:09:40 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:09:40.902+0000 7fbcbd9ce000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-24T11:09:41.136 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:09:40 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:09:40.986+0000 7fbcbd9ce000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-24T11:09:41.413 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:09:41 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:09:41.135+0000 7fbcbd9ce000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-24T11:09:41.413 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:09:41 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:09:41.217+0000 7fbcbd9ce000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-24T11:09:41.413 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:09:41 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:09:41.328+0000 7fbcbd9ce000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-24T11:09:41.413 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:09:41 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:09:41.412+0000 7fbcbd9ce000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-24T11:09:41.726 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:09:41 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:09:41.575+0000 7fbcbd9ce000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-24T11:09:41.726 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:09:41 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:09:41.662+0000 7fbcbd9ce000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-24T11:09:42.977 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:09:42 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:09:42.718+0000 7fbcbd9ce000 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-24T11:09:42.977 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:09:42 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:09:42.854+0000 7fbcbd9ce000 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-24T11:09:43.279 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:09:43 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:09:43.049+0000 7fbcbd9ce000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-24T11:09:43.279 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:09:43 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:09:43.138+0000 7fbcbd9ce000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-24T11:09:43.279 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:09:43 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:09:43.278+0000 7fbcbd9ce000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-24T11:09:43.312 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: 2024-03-24T11:09:43.312 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: { 2024-03-24T11:09:43.312 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "fsid": "ebb89568-e9ce-11ee-95cd-87774f69a715", 2024-03-24T11:09:43.312 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "health": { 2024-03-24T11:09:43.312 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "status": "HEALTH_OK", 2024-03-24T11:09:43.313 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "checks": {}, 2024-03-24T11:09:43.313 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "mutes": [] 2024-03-24T11:09:43.313 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: }, 2024-03-24T11:09:43.313 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "election_epoch": 5, 2024-03-24T11:09:43.313 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "quorum": [ 2024-03-24T11:09:43.313 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: 0 2024-03-24T11:09:43.313 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: ], 2024-03-24T11:09:43.313 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "quorum_names": [ 2024-03-24T11:09:43.313 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "a" 2024-03-24T11:09:43.313 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: ], 2024-03-24T11:09:43.313 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "quorum_age": 5, 2024-03-24T11:09:43.313 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "monmap": { 2024-03-24T11:09:43.313 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "epoch": 1, 2024-03-24T11:09:43.313 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "min_mon_release_name": "quincy", 2024-03-24T11:09:43.313 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "num_mons": 1 2024-03-24T11:09:43.314 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: }, 2024-03-24T11:09:43.314 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "osdmap": { 2024-03-24T11:09:43.314 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "epoch": 1, 2024-03-24T11:09:43.314 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "num_osds": 0, 2024-03-24T11:09:43.314 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "num_up_osds": 0, 2024-03-24T11:09:43.314 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "osd_up_since": 0, 2024-03-24T11:09:43.314 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "num_in_osds": 0, 2024-03-24T11:09:43.314 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "osd_in_since": 0, 2024-03-24T11:09:43.314 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "num_remapped_pgs": 0 2024-03-24T11:09:43.314 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: }, 2024-03-24T11:09:43.314 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "pgmap": { 2024-03-24T11:09:43.315 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "pgs_by_state": [], 2024-03-24T11:09:43.315 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "num_pgs": 0, 2024-03-24T11:09:43.315 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "num_pools": 0, 2024-03-24T11:09:43.315 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "num_objects": 0, 2024-03-24T11:09:43.315 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "data_bytes": 0, 2024-03-24T11:09:43.315 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "bytes_used": 0, 2024-03-24T11:09:43.315 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "bytes_avail": 0, 2024-03-24T11:09:43.316 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "bytes_total": 0 2024-03-24T11:09:43.316 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: }, 2024-03-24T11:09:43.316 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "fsmap": { 2024-03-24T11:09:43.316 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "epoch": 1, 2024-03-24T11:09:43.316 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "by_rank": [], 2024-03-24T11:09:43.316 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "up:standby": 0 2024-03-24T11:09:43.316 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: }, 2024-03-24T11:09:43.316 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "mgrmap": { 2024-03-24T11:09:43.316 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "available": false, 2024-03-24T11:09:43.316 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "num_standbys": 0, 2024-03-24T11:09:43.316 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "modules": [ 2024-03-24T11:09:43.316 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "iostat", 2024-03-24T11:09:43.316 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "nfs", 2024-03-24T11:09:43.316 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "restful" 2024-03-24T11:09:43.316 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: ], 2024-03-24T11:09:43.317 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "services": {} 2024-03-24T11:09:43.317 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: }, 2024-03-24T11:09:43.317 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "servicemap": { 2024-03-24T11:09:43.317 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "epoch": 1, 2024-03-24T11:09:43.317 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "modified": "2024-03-24T11:09:33.575913+0000", 2024-03-24T11:09:43.317 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "services": {} 2024-03-24T11:09:43.317 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: }, 2024-03-24T11:09:43.317 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "progress_events": {} 2024-03-24T11:09:43.317 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: } 2024-03-24T11:09:43.911 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:43 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/63777463' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-24T11:09:43.912 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:09:43 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:09:43.600+0000 7fbcbd9ce000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-24T11:09:43.912 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:09:43 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:09:43.825+0000 7fbcbd9ce000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-24T11:09:43.912 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:09:43 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:09:43.911+0000 7fbcbd9ce000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-24T11:09:44.226 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:09:44 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:09:44.085+0000 7fbcbd9ce000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-24T11:09:45.323 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:09:45 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:09:45.023+0000 7fbcbd9ce000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-24T11:09:45.323 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:09:45 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:09:45.322+0000 7fbcbd9ce000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-24T11:09:45.634 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:09:45 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:09:45.418+0000 7fbcbd9ce000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-24T11:09:45.634 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:09:45 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:09:45.569+0000 7fbcbd9ce000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-24T11:09:45.976 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:45 smithi062 ceph-mon[24799]: Activating manager daemon y 2024-03-24T11:09:45.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:45 smithi062 ceph-mon[24799]: mgrmap e2: y(active, starting, since 0.0026683s) 2024-03-24T11:09:45.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:45 smithi062 ceph-mon[24799]: from='mgr.14100 172.21.15.62:0/1240875848' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-24T11:09:45.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:45 smithi062 ceph-mon[24799]: from='mgr.14100 172.21.15.62:0/1240875848' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-24T11:09:45.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:45 smithi062 ceph-mon[24799]: from='mgr.14100 172.21.15.62:0/1240875848' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-24T11:09:45.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:45 smithi062 ceph-mon[24799]: from='mgr.14100 172.21.15.62:0/1240875848' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T11:09:45.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:45 smithi062 ceph-mon[24799]: from='mgr.14100 172.21.15.62:0/1240875848' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-24T11:09:45.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:45 smithi062 ceph-mon[24799]: Manager daemon y is now available 2024-03-24T11:09:45.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:45 smithi062 ceph-mon[24799]: from='mgr.14100 172.21.15.62:0/1240875848' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-24T11:09:45.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:45 smithi062 ceph-mon[24799]: from='mgr.14100 172.21.15.62:0/1240875848' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-24T11:09:45.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:45 smithi062 ceph-mon[24799]: from='mgr.14100 172.21.15.62:0/1240875848' entity='mgr.y' 2024-03-24T11:09:45.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:45 smithi062 ceph-mon[24799]: from='mgr.14100 172.21.15.62:0/1240875848' entity='mgr.y' 2024-03-24T11:09:45.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:45 smithi062 ceph-mon[24799]: from='mgr.14100 172.21.15.62:0/1240875848' entity='mgr.y' 2024-03-24T11:09:47.976 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:47 smithi062 ceph-mon[24799]: mgrmap e3: y(active, since 1.0069s) 2024-03-24T11:09:48.455 INFO:teuthology.orchestra.run.smithi062.stderr:mgr not available, waiting (2/15)... 2024-03-24T11:09:49.726 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:49 smithi062 ceph-mon[24799]: mgrmap e4: y(active, since 3s) 2024-03-24T11:09:52.397 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: 2024-03-24T11:09:52.397 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: { 2024-03-24T11:09:52.397 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "fsid": "ebb89568-e9ce-11ee-95cd-87774f69a715", 2024-03-24T11:09:52.398 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "health": { 2024-03-24T11:09:52.398 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "status": "HEALTH_OK", 2024-03-24T11:09:52.398 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "checks": {}, 2024-03-24T11:09:52.398 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "mutes": [] 2024-03-24T11:09:52.398 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: }, 2024-03-24T11:09:52.398 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "election_epoch": 5, 2024-03-24T11:09:52.398 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "quorum": [ 2024-03-24T11:09:52.398 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: 0 2024-03-24T11:09:52.398 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: ], 2024-03-24T11:09:52.398 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "quorum_names": [ 2024-03-24T11:09:52.398 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "a" 2024-03-24T11:09:52.398 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: ], 2024-03-24T11:09:52.398 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "quorum_age": 14, 2024-03-24T11:09:52.398 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "monmap": { 2024-03-24T11:09:52.399 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "epoch": 1, 2024-03-24T11:09:52.399 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "min_mon_release_name": "quincy", 2024-03-24T11:09:52.399 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "num_mons": 1 2024-03-24T11:09:52.399 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: }, 2024-03-24T11:09:52.399 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "osdmap": { 2024-03-24T11:09:52.399 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "epoch": 1, 2024-03-24T11:09:52.399 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "num_osds": 0, 2024-03-24T11:09:52.399 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "num_up_osds": 0, 2024-03-24T11:09:52.399 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "osd_up_since": 0, 2024-03-24T11:09:52.399 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "num_in_osds": 0, 2024-03-24T11:09:52.403 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "osd_in_since": 0, 2024-03-24T11:09:52.403 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "num_remapped_pgs": 0 2024-03-24T11:09:52.403 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: }, 2024-03-24T11:09:52.403 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "pgmap": { 2024-03-24T11:09:52.403 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "pgs_by_state": [], 2024-03-24T11:09:52.403 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "num_pgs": 0, 2024-03-24T11:09:52.403 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "num_pools": 0, 2024-03-24T11:09:52.403 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "num_objects": 0, 2024-03-24T11:09:52.403 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "data_bytes": 0, 2024-03-24T11:09:52.403 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "bytes_used": 0, 2024-03-24T11:09:52.403 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "bytes_avail": 0, 2024-03-24T11:09:52.403 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "bytes_total": 0 2024-03-24T11:09:52.403 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: }, 2024-03-24T11:09:52.404 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "fsmap": { 2024-03-24T11:09:52.404 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "epoch": 1, 2024-03-24T11:09:52.404 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "by_rank": [], 2024-03-24T11:09:52.404 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "up:standby": 0 2024-03-24T11:09:52.404 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: }, 2024-03-24T11:09:52.404 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "mgrmap": { 2024-03-24T11:09:52.404 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "available": true, 2024-03-24T11:09:52.404 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "num_standbys": 0, 2024-03-24T11:09:52.404 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "modules": [ 2024-03-24T11:09:52.404 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "iostat", 2024-03-24T11:09:52.404 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "nfs", 2024-03-24T11:09:52.404 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "restful" 2024-03-24T11:09:52.404 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: ], 2024-03-24T11:09:52.404 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "services": {} 2024-03-24T11:09:52.404 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: }, 2024-03-24T11:09:52.405 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "servicemap": { 2024-03-24T11:09:52.405 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "epoch": 1, 2024-03-24T11:09:52.405 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "modified": "2024-03-24T11:09:33.575913+0000", 2024-03-24T11:09:52.405 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "services": {} 2024-03-24T11:09:52.405 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: }, 2024-03-24T11:09:52.405 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "progress_events": {} 2024-03-24T11:09:52.405 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: } 2024-03-24T11:09:52.726 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:52 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/4129121563' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-24T11:09:52.977 INFO:teuthology.orchestra.run.smithi062.stderr:mgr is available 2024-03-24T11:09:53.565 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: 2024-03-24T11:09:53.565 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: [global] 2024-03-24T11:09:53.565 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: fsid = ebb89568-e9ce-11ee-95cd-87774f69a715 2024-03-24T11:09:53.565 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: mon_osd_allow_pg_remap = true 2024-03-24T11:09:53.565 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: mon_osd_allow_primary_affinity = true 2024-03-24T11:09:53.566 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: mon_warn_on_no_sortbitwise = false 2024-03-24T11:09:53.566 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: osd_crush_chooseleaf_type = 0 2024-03-24T11:09:53.566 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: 2024-03-24T11:09:53.566 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: [mgr] 2024-03-24T11:09:53.566 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: mgr/telemetry/nag = false 2024-03-24T11:09:53.566 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: 2024-03-24T11:09:53.566 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: [osd] 2024-03-24T11:09:53.566 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: osd_map_max_advance = 10 2024-03-24T11:09:53.566 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: osd_sloppy_crc = true 2024-03-24T11:09:54.056 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:53 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/1543213108' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-24T11:09:54.056 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:53 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/1543213108' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-24T11:09:54.059 INFO:teuthology.orchestra.run.smithi062.stderr:Enabling cephadm module... 2024-03-24T11:09:55.070 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:54 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/4058441783' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-24T11:09:55.070 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:09:54 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: ignoring --setuser ceph since I am not root 2024-03-24T11:09:55.070 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:09:54 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: ignoring --setgroup ceph since I am not root 2024-03-24T11:09:55.375 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:09:55 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:09:55.277+0000 7f8ab07a9000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-24T11:09:55.375 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:09:55 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:09:55.374+0000 7f8ab07a9000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-24T11:09:55.955 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:55 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/4058441783' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-24T11:09:55.956 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:55 smithi062 ceph-mon[24799]: mgrmap e5: y(active, since 9s) 2024-03-24T11:09:55.956 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: { 2024-03-24T11:09:55.956 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "epoch": 5, 2024-03-24T11:09:55.956 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "available": true, 2024-03-24T11:09:55.957 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "active_name": "y", 2024-03-24T11:09:55.957 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "num_standby": 0 2024-03-24T11:09:55.957 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: } 2024-03-24T11:09:56.226 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:09:55 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:09:55.989+0000 7f8ab07a9000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-24T11:09:56.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:09:56 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:09:56.079+0000 7f8ab07a9000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-24T11:09:56.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:09:56 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:09:56.163+0000 7f8ab07a9000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-24T11:09:56.560 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:09:56 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:09:56.314+0000 7f8ab07a9000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-24T11:09:56.561 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:09:56 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:09:56.396+0000 7f8ab07a9000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-24T11:09:56.561 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:09:56 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:09:56.506+0000 7f8ab07a9000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-24T11:09:56.563 INFO:teuthology.orchestra.run.smithi062.stderr:Waiting for the mgr to restart... 2024-03-24T11:09:56.563 INFO:teuthology.orchestra.run.smithi062.stderr:Waiting for mgr epoch 5... 2024-03-24T11:09:56.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:09:56 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/3559019997' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-24T11:09:56.858 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:09:56 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:09:56.601+0000 7f8ab07a9000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-24T11:09:56.858 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:09:56 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:09:56.766+0000 7f8ab07a9000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-24T11:09:56.858 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:09:56 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:09:56.857+0000 7f8ab07a9000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-24T11:09:58.226 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:09:57 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:09:57.921+0000 7f8ab07a9000 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-24T11:09:58.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:09:58 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:09:58.055+0000 7f8ab07a9000 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-24T11:09:58.726 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:09:58 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:09:58.247+0000 7f8ab07a9000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-24T11:09:58.726 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:09:58 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:09:58.334+0000 7f8ab07a9000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-24T11:09:58.727 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:09:58 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:09:58.471+0000 7f8ab07a9000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-24T11:09:59.100 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:09:58 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:09:58.791+0000 7f8ab07a9000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-24T11:09:59.100 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:09:59 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:09:59.014+0000 7f8ab07a9000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-24T11:09:59.100 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:09:59 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:09:59.099+0000 7f8ab07a9000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-24T11:09:59.476 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:09:59 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:09:59.271+0000 7f8ab07a9000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-24T11:10:00.476 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:00 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:10:00.199+0000 7f8ab07a9000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-24T11:10:00.812 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:00 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:10:00.498+0000 7f8ab07a9000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-24T11:10:00.812 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:00 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:10:00.594+0000 7f8ab07a9000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-24T11:10:00.813 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:00 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:10:00.746+0000 7f8ab07a9000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-24T11:10:01.091 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:00 smithi062 ceph-mon[24799]: Active manager daemon y restarted 2024-03-24T11:10:01.091 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:00 smithi062 ceph-mon[24799]: Activating manager daemon y 2024-03-24T11:10:01.091 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:00 smithi062 ceph-mon[24799]: osdmap e2: 0 total, 0 up, 0 in 2024-03-24T11:10:01.831 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: { 2024-03-24T11:10:01.831 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "mgrmap_epoch": 7, 2024-03-24T11:10:01.831 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "initialized": true 2024-03-24T11:10:01.832 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: } 2024-03-24T11:10:02.064 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:01 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: [24/Mar/2024:11:10:01] ENGINE Bus STARTING 2024-03-24T11:10:02.065 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:02 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: [24/Mar/2024:11:10:02] ENGINE Serving on https://172.21.15.62:7150 2024-03-24T11:10:02.065 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:01 smithi062 ceph-mon[24799]: mgrmap e6: y(active, starting, since 0.0525454s) 2024-03-24T11:10:02.065 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:01 smithi062 ceph-mon[24799]: from='mgr.14118 172.21.15.62:0/3591406410' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T11:10:02.065 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:01 smithi062 ceph-mon[24799]: from='mgr.14118 172.21.15.62:0/3591406410' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-24T11:10:02.065 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:01 smithi062 ceph-mon[24799]: from='mgr.14118 172.21.15.62:0/3591406410' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-24T11:10:02.065 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:01 smithi062 ceph-mon[24799]: from='mgr.14118 172.21.15.62:0/3591406410' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-24T11:10:02.066 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:01 smithi062 ceph-mon[24799]: from='mgr.14118 172.21.15.62:0/3591406410' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-24T11:10:02.066 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:01 smithi062 ceph-mon[24799]: Manager daemon y is now available 2024-03-24T11:10:02.066 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:01 smithi062 ceph-mon[24799]: from='mgr.14118 172.21.15.62:0/3591406410' entity='mgr.y' 2024-03-24T11:10:02.066 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:01 smithi062 ceph-mon[24799]: from='mgr.14118 172.21.15.62:0/3591406410' entity='mgr.y' 2024-03-24T11:10:02.066 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:01 smithi062 ceph-mon[24799]: from='mgr.14118 172.21.15.62:0/3591406410' entity='mgr.y' 2024-03-24T11:10:02.066 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:01 smithi062 ceph-mon[24799]: from='mgr.14118 172.21.15.62:0/3591406410' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:10:02.066 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:01 smithi062 ceph-mon[24799]: from='mgr.14118 172.21.15.62:0/3591406410' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:10:02.066 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:01 smithi062 ceph-mon[24799]: from='mgr.14118 172.21.15.62:0/3591406410' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:10:02.066 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:01 smithi062 ceph-mon[24799]: from='mgr.14118 172.21.15.62:0/3591406410' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-24T11:10:02.066 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:01 smithi062 ceph-mon[24799]: from='mgr.14118 172.21.15.62:0/3591406410' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-24T11:10:02.367 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:02 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: [24/Mar/2024:11:10:02] ENGINE Bus STARTED 2024-03-24T11:10:02.369 INFO:teuthology.orchestra.run.smithi062.stderr:mgr epoch 5 is available 2024-03-24T11:10:02.369 INFO:teuthology.orchestra.run.smithi062.stderr:Setting orchestrator backend to cephadm... 2024-03-24T11:10:02.976 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:02 smithi062 ceph-mon[24799]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-24T11:10:02.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:02 smithi062 ceph-mon[24799]: mgrmap e7: y(active, since 1.05971s) 2024-03-24T11:10:02.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:02 smithi062 ceph-mon[24799]: from='mgr.14118 172.21.15.62:0/3591406410' entity='mgr.y' 2024-03-24T11:10:02.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:02 smithi062 ceph-mon[24799]: from='mgr.14118 172.21.15.62:0/3591406410' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:10:04.142 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:03 smithi062 ceph-mon[24799]: [24/Mar/2024:11:10:01] ENGINE Bus STARTING 2024-03-24T11:10:04.143 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:03 smithi062 ceph-mon[24799]: [24/Mar/2024:11:10:02] ENGINE Serving on https://172.21.15.62:7150 2024-03-24T11:10:04.143 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:03 smithi062 ceph-mon[24799]: [24/Mar/2024:11:10:02] ENGINE Bus STARTED 2024-03-24T11:10:04.143 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:03 smithi062 ceph-mon[24799]: from='mgr.14118 172.21.15.62:0/3591406410' entity='mgr.y' 2024-03-24T11:10:04.143 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:03 smithi062 ceph-mon[24799]: from='mgr.14118 172.21.15.62:0/3591406410' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:10:04.143 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:03 smithi062 ceph-mon[24799]: mgrmap e8: y(active, since 2s) 2024-03-24T11:10:04.143 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: value unchanged 2024-03-24T11:10:04.706 INFO:teuthology.orchestra.run.smithi062.stderr:Generating ssh key... 2024-03-24T11:10:04.976 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:04 smithi062 ceph-mon[24799]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:10:05.819 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:05 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: Generating public/private rsa key pair. 2024-03-24T11:10:05.819 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:05 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: Your identification has been saved in /tmp/tmpvs7694l8/key. 2024-03-24T11:10:05.819 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:05 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: Your public key has been saved in /tmp/tmpvs7694l8/key.pub. 2024-03-24T11:10:05.819 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:05 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: The key fingerprint is: 2024-03-24T11:10:05.819 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:05 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: SHA256:cvyXPa70otwXg+7BYHF2t37HxzRrpghjMmmN1AWCRSs ceph-ebb89568-e9ce-11ee-95cd-87774f69a715 2024-03-24T11:10:05.820 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:05 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: The key's randomart image is: 2024-03-24T11:10:05.820 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:05 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: +---[RSA 3072]----+ 2024-03-24T11:10:05.820 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:05 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: | ++ . | 2024-03-24T11:10:05.820 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:05 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: | . o . | 2024-03-24T11:10:05.820 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:05 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: | E . ..o . .| 2024-03-24T11:10:05.820 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:05 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: | ... .+ . ..| 2024-03-24T11:10:05.820 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:05 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: | ..S.o . o.| 2024-03-24T11:10:05.820 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:05 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: | .o+o o.o++o| 2024-03-24T11:10:05.820 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:05 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: | * =..* oB*| 2024-03-24T11:10:05.820 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:05 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: | . +.o+++=.+| 2024-03-24T11:10:05.820 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:05 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: | o++=o | 2024-03-24T11:10:05.820 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:05 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: +----[SHA256]-----+ 2024-03-24T11:10:06.118 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:05 smithi062 ceph-mon[24799]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:10:06.118 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:05 smithi062 ceph-mon[24799]: from='mgr.14118 172.21.15.62:0/3591406410' entity='mgr.y' 2024-03-24T11:10:06.118 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:05 smithi062 ceph-mon[24799]: from='mgr.14118 172.21.15.62:0/3591406410' entity='mgr.y' 2024-03-24T11:10:06.699 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7P15b6kerwdAGK23zLWG6/zhUInHzJUYiQf9oQS3PmFYAMgu8ku/hdxcSo1Ekd0DIraph+boKnSlPILHo8ZuOwlYiX7OrWlvsiFUDzijK21wGhDdjY9dXxKuKRLaDMSpFUZDMW8BHCdP746jqg+hPfzK7krzOt9lpONIkFHJCDVSPiTSv5gGqLvGiS+PfFs0TnQlWnHZhq2hJRDcwwpfAp5LZ9vTwIoEL3YMei6RcgRPkZRugQ1hXy9fXjvpXqtlWfWO9RhN3uL+IjT8idQoOFF9PF4/dZsGs1VWR/TkaiXUMUPSyucWzE4PKTyazp/xEssRAsU+Dsk6aI4oHmYAQRWvkI6izEVW5froDVDEYE4ollgvo32TpOSPrUHOMxEXIUlh84Gqk5rop+zfQ5YNM7KzQAna/iuTg3b3SoF64cv9HlE3AHPHHMAPT/fvwttB5i2nsFALh3zRlYlwqeyN3Wvhkj8e4YKAvqe3P1Npo97110MKIYd3Eh5qJa2vAfQE= ceph-ebb89568-e9ce-11ee-95cd-87774f69a715 2024-03-24T11:10:06.976 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:06 smithi062 ceph-mon[24799]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:10:06.976 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:06 smithi062 ceph-mon[24799]: Generating ssh key... 2024-03-24T11:10:07.236 INFO:teuthology.orchestra.run.smithi062.stderr:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-24T11:10:07.236 INFO:teuthology.orchestra.run.smithi062.stderr:Adding key to root@localhost authorized_keys... 2024-03-24T11:10:07.236 INFO:teuthology.orchestra.run.smithi062.stderr:Adding host smithi062... 2024-03-24T11:10:08.133 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:07 smithi062 ceph-mon[24799]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:10:08.835 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:08 smithi062 ceph-mon[24799]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi062", "addr": "172.21.15.62", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:10:08.836 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: Added host 'smithi062' with addr '172.21.15.62' 2024-03-24T11:10:09.435 INFO:teuthology.orchestra.run.smithi062.stderr:Deploying unmanaged mon service... 2024-03-24T11:10:10.058 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:09 smithi062 ceph-mon[24799]: Deploying cephadm binary to smithi062 2024-03-24T11:10:10.058 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:09 smithi062 ceph-mon[24799]: from='mgr.14118 172.21.15.62:0/3591406410' entity='mgr.y' 2024-03-24T11:10:10.058 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:09 smithi062 ceph-mon[24799]: from='mgr.14118 172.21.15.62:0/3591406410' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:10:10.058 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: Scheduled mon update... 2024-03-24T11:10:10.759 INFO:teuthology.orchestra.run.smithi062.stderr:Deploying unmanaged mgr service... 2024-03-24T11:10:11.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:10 smithi062 ceph-mon[24799]: Added host smithi062 2024-03-24T11:10:11.085 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:10 smithi062 ceph-mon[24799]: from='mgr.14118 172.21.15.62:0/3591406410' entity='mgr.y' 2024-03-24T11:10:11.932 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:11 smithi062 ceph-mon[24799]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:10:11.933 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:11 smithi062 ceph-mon[24799]: Saving service mon spec with placement count:5 2024-03-24T11:10:12.325 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: Scheduled mgr update... 2024-03-24T11:10:13.181 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:13 smithi062 ceph-mon[24799]: from='mgr.14118 172.21.15.62:0/3591406410' entity='mgr.y' 2024-03-24T11:10:13.181 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:13 smithi062 ceph-mon[24799]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:10:13.181 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:13 smithi062 ceph-mon[24799]: Saving service mgr spec with placement count:2 2024-03-24T11:10:13.181 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:13 smithi062 ceph-mon[24799]: from='mgr.14118 172.21.15.62:0/3591406410' entity='mgr.y' 2024-03-24T11:10:13.181 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:13 smithi062 ceph-mon[24799]: from='mgr.14118 172.21.15.62:0/3591406410' entity='mgr.y' 2024-03-24T11:10:14.849 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:14 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/4016972719' entity='client.admin' 2024-03-24T11:10:14.849 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:14 smithi062 ceph-mon[24799]: from='mgr.14118 172.21.15.62:0/3591406410' entity='mgr.y' 2024-03-24T11:10:14.849 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:14 smithi062 ceph-mon[24799]: from='mgr.14118 172.21.15.62:0/3591406410' entity='mgr.y' 2024-03-24T11:10:14.849 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:14 smithi062 ceph-mon[24799]: from='mgr.14118 172.21.15.62:0/3591406410' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi062", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:10:14.850 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:14 smithi062 ceph-mon[24799]: from='mgr.14118 172.21.15.62:0/3591406410' entity='mgr.y' 2024-03-24T11:10:14.850 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:14 smithi062 ceph-mon[24799]: from='mgr.14118 172.21.15.62:0/3591406410' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi062", "caps": []}]: dispatch 2024-03-24T11:10:14.850 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:14 smithi062 ceph-mon[24799]: from='mgr.14118 172.21.15.62:0/3591406410' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.agent.smithi062", "caps": []}]': finished 2024-03-24T11:10:15.688 INFO:teuthology.orchestra.run.smithi062.stderr:Enabling the dashboard module... 2024-03-24T11:10:16.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:16 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/4080977500' entity='client.admin' 2024-03-24T11:10:16.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:16 smithi062 ceph-mon[24799]: Deploying daemon agent.smithi062 on smithi062 2024-03-24T11:10:17.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:17 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/377264845' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-24T11:10:17.477 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:17 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: ignoring --setuser ceph since I am not root 2024-03-24T11:10:17.477 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:17 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: ignoring --setgroup ceph since I am not root 2024-03-24T11:10:17.851 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:17 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:10:17.655+0000 7f7befb4d000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-24T11:10:17.851 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:17 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:10:17.745+0000 7f7befb4d000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-24T11:10:18.454 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:18 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/377264845' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-24T11:10:18.455 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:18 smithi062 ceph-mon[24799]: mgrmap e9: y(active, since 16s) 2024-03-24T11:10:18.455 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:18 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:10:18.363+0000 7f7befb4d000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-24T11:10:18.455 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:18 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:10:18.454+0000 7f7befb4d000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-24T11:10:18.509 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: { 2024-03-24T11:10:18.509 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "epoch": 9, 2024-03-24T11:10:18.509 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "available": true, 2024-03-24T11:10:18.509 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "active_name": "y", 2024-03-24T11:10:18.509 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "num_standby": 0 2024-03-24T11:10:18.509 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: } 2024-03-24T11:10:18.726 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:18 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:10:18.546+0000 7f7befb4d000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-24T11:10:18.726 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:18 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:10:18.698+0000 7f7befb4d000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-24T11:10:19.099 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:18 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:10:18.781+0000 7f7befb4d000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-24T11:10:19.100 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:18 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:10:18.891+0000 7f7befb4d000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-24T11:10:19.100 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:18 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:10:18.976+0000 7f7befb4d000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-24T11:10:19.102 INFO:teuthology.orchestra.run.smithi062.stderr:Waiting for the mgr to restart... 2024-03-24T11:10:19.102 INFO:teuthology.orchestra.run.smithi062.stderr:Waiting for mgr epoch 9... 2024-03-24T11:10:19.452 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:19 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/4192740819' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-24T11:10:19.453 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:19 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:10:19.144+0000 7f7befb4d000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-24T11:10:19.453 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:19 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:10:19.231+0000 7f7befb4d000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-24T11:10:20.602 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:20 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:10:20.275+0000 7f7befb4d000 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-24T11:10:20.602 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:20 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:10:20.407+0000 7f7befb4d000 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-24T11:10:20.602 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:20 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:10:20.601+0000 7f7befb4d000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-24T11:10:20.976 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:20 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:10:20.690+0000 7f7befb4d000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-24T11:10:20.976 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:20 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:10:20.828+0000 7f7befb4d000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-24T11:10:21.460 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:21 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:10:21.148+0000 7f7befb4d000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-24T11:10:21.460 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:21 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:10:21.374+0000 7f7befb4d000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-24T11:10:21.460 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:21 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:10:21.460+0000 7f7befb4d000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-24T11:10:21.726 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:21 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:10:21.633+0000 7f7befb4d000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-24T11:10:22.866 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:22 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:10:22.565+0000 7f7befb4d000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-24T11:10:22.867 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:22 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:10:22.866+0000 7f7befb4d000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-24T11:10:23.226 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:22 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:10:22.961+0000 7f7befb4d000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-24T11:10:23.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:23 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:10:23.115+0000 7f7befb4d000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-24T11:10:24.039 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:23 smithi062 ceph-mon[24799]: Active manager daemon y restarted 2024-03-24T11:10:24.039 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:23 smithi062 ceph-mon[24799]: Activating manager daemon y 2024-03-24T11:10:24.040 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:23 smithi062 ceph-mon[24799]: osdmap e3: 0 total, 0 up, 0 in 2024-03-24T11:10:24.700 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: [24/Mar/2024:11:10:24] ENGINE Bus STARTING 2024-03-24T11:10:24.700 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: [24/Mar/2024:11:10:24] ENGINE Serving on https://172.21.15.62:7150 2024-03-24T11:10:24.700 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: [24/Mar/2024:11:10:24] ENGINE Bus STARTED 2024-03-24T11:10:24.798 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: { 2024-03-24T11:10:24.799 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "mgrmap_epoch": 11, 2024-03-24T11:10:24.799 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: "initialized": true 2024-03-24T11:10:24.799 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: } 2024-03-24T11:10:24.976 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:24 smithi062 ceph-mon[24799]: mgrmap e10: y(active, starting, since 0.657573s) 2024-03-24T11:10:24.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:24 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T11:10:24.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:24 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-24T11:10:24.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:24 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-24T11:10:24.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:24 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-24T11:10:24.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:24 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-24T11:10:24.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:24 smithi062 ceph-mon[24799]: Manager daemon y is now available 2024-03-24T11:10:24.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:24 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:10:24.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:24 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:10:24.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:24 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:10:24.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:24 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-24T11:10:24.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:24 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-24T11:10:24.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:24 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:10:25.554 INFO:teuthology.orchestra.run.smithi062.stderr:mgr epoch 9 is available 2024-03-24T11:10:25.554 INFO:teuthology.orchestra.run.smithi062.stderr:Generating a dashboard self-signed certificate... 2024-03-24T11:10:25.976 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:25 smithi062 ceph-mon[24799]: [24/Mar/2024:11:10:24] ENGINE Bus STARTING 2024-03-24T11:10:25.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:25 smithi062 ceph-mon[24799]: [24/Mar/2024:11:10:24] ENGINE Serving on https://172.21.15.62:7150 2024-03-24T11:10:25.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:25 smithi062 ceph-mon[24799]: [24/Mar/2024:11:10:24] ENGINE Bus STARTED 2024-03-24T11:10:25.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:25 smithi062 ceph-mon[24799]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-24T11:10:25.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:25 smithi062 ceph-mon[24799]: mgrmap e11: y(active, since 1.663s) 2024-03-24T11:10:25.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:25 smithi062 ceph-mon[24799]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-24T11:10:25.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:25 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:10:25.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:25 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi062", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:10:25.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:25 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:10:25.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:25 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi062", "caps": []}]: dispatch 2024-03-24T11:10:26.335 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: Self-signed certificate created 2024-03-24T11:10:26.892 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:26 smithi062 ceph-mon[24799]: mgrmap e12: y(active, since 2s) 2024-03-24T11:10:26.892 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:26 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:10:26.892 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:26 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:10:26.894 INFO:teuthology.orchestra.run.smithi062.stderr:Creating initial admin user... 2024-03-24T11:10:27.851 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:27 smithi062 ceph-mon[24799]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:10:27.851 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:27 smithi062 ceph-mon[24799]: Deploying daemon agent.smithi062 on smithi062 2024-03-24T11:10:27.851 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: {"username": "admin", "password": "$2b$12$mQLLzFWrpFvhuShPHNOSz.ug3R6gvtF3oZoQv.nBD7XMhOjAR/Hgi", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1711278627, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-24T11:10:28.502 INFO:teuthology.orchestra.run.smithi062.stderr:Fetching dashboard port number... 2024-03-24T11:10:29.163 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:28 smithi062 ceph-mon[24799]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:10:29.163 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:28 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:10:29.163 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: 8443 2024-03-24T11:10:29.760 INFO:teuthology.orchestra.run.smithi062.stderr:systemctl: disabled 2024-03-24T11:10:29.766 INFO:teuthology.orchestra.run.smithi062.stderr:systemctl: inactive 2024-03-24T11:10:29.767 INFO:teuthology.orchestra.run.smithi062.stderr:firewalld.service is not enabled 2024-03-24T11:10:29.767 INFO:teuthology.orchestra.run.smithi062.stderr:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-24T11:10:29.769 INFO:teuthology.orchestra.run.smithi062.stderr:Ceph Dashboard is now available at: 2024-03-24T11:10:29.769 INFO:teuthology.orchestra.run.smithi062.stderr: 2024-03-24T11:10:29.769 INFO:teuthology.orchestra.run.smithi062.stderr: URL: https://smithi062.front.sepia.ceph.com:8443/ 2024-03-24T11:10:29.770 INFO:teuthology.orchestra.run.smithi062.stderr: User: admin 2024-03-24T11:10:29.770 INFO:teuthology.orchestra.run.smithi062.stderr: Password: 0tpaxrahzd 2024-03-24T11:10:29.770 INFO:teuthology.orchestra.run.smithi062.stderr: 2024-03-24T11:10:29.770 INFO:teuthology.orchestra.run.smithi062.stderr:Enabling autotune for osd_memory_target 2024-03-24T11:10:30.033 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:29 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/2656436312' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-24T11:10:31.072 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:30 smithi062 ceph-mon[24799]: mgrmap e13: y(active, since 6s) 2024-03-24T11:10:31.831 INFO:teuthology.orchestra.run.smithi062.stderr:/usr/bin/ceph: set mgr/dashboard/cluster/status 2024-03-24T11:10:32.431 INFO:teuthology.orchestra.run.smithi062.stderr:You can access the Ceph CLI with: 2024-03-24T11:10:32.431 INFO:teuthology.orchestra.run.smithi062.stderr: 2024-03-24T11:10:32.431 INFO:teuthology.orchestra.run.smithi062.stderr: sudo /home/ubuntu/cephtest/cephadm shell --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-24T11:10:32.431 INFO:teuthology.orchestra.run.smithi062.stderr: 2024-03-24T11:10:32.431 INFO:teuthology.orchestra.run.smithi062.stderr:Please consider enabling telemetry to help improve Ceph: 2024-03-24T11:10:32.431 INFO:teuthology.orchestra.run.smithi062.stderr: 2024-03-24T11:10:32.431 INFO:teuthology.orchestra.run.smithi062.stderr: ceph telemetry on 2024-03-24T11:10:32.431 INFO:teuthology.orchestra.run.smithi062.stderr: 2024-03-24T11:10:32.431 INFO:teuthology.orchestra.run.smithi062.stderr:For more information see: 2024-03-24T11:10:32.432 INFO:teuthology.orchestra.run.smithi062.stderr: 2024-03-24T11:10:32.432 INFO:teuthology.orchestra.run.smithi062.stderr: https://docs.ceph.com/docs/master/mgr/telemetry/ 2024-03-24T11:10:32.432 INFO:teuthology.orchestra.run.smithi062.stderr: 2024-03-24T11:10:32.432 INFO:teuthology.orchestra.run.smithi062.stderr:Bootstrap complete. 2024-03-24T11:10:32.613 INFO:tasks.cephadm:Fetching config... 2024-03-24T11:10:32.613 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-24T11:10:32.613 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-24T11:10:32.628 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-24T11:10:32.628 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-24T11:10:32.628 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-24T11:10:32.686 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-24T11:10:32.686 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-24T11:10:32.686 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/mon.a/keyring of=/dev/stdout 2024-03-24T11:10:32.806 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-24T11:10:32.806 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-24T11:10:32.806 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-24T11:10:32.863 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-24T11:10:32.864 DEBUG:teuthology.orchestra.run.smithi062:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7P15b6kerwdAGK23zLWG6/zhUInHzJUYiQf9oQS3PmFYAMgu8ku/hdxcSo1Ekd0DIraph+boKnSlPILHo8ZuOwlYiX7OrWlvsiFUDzijK21wGhDdjY9dXxKuKRLaDMSpFUZDMW8BHCdP746jqg+hPfzK7krzOt9lpONIkFHJCDVSPiTSv5gGqLvGiS+PfFs0TnQlWnHZhq2hJRDcwwpfAp5LZ9vTwIoEL3YMei6RcgRPkZRugQ1hXy9fXjvpXqtlWfWO9RhN3uL+IjT8idQoOFF9PF4/dZsGs1VWR/TkaiXUMUPSyucWzE4PKTyazp/xEssRAsU+Dsk6aI4oHmYAQRWvkI6izEVW5froDVDEYE4ollgvo32TpOSPrUHOMxEXIUlh84Gqk5rop+zfQ5YNM7KzQAna/iuTg3b3SoF64cv9HlE3AHPHHMAPT/fvwttB5i2nsFALh3zRlYlwqeyN3Wvhkj8e4YKAvqe3P1Npo97110MKIYd3Eh5qJa2vAfQE= ceph-ebb89568-e9ce-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-24T11:10:32.874 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:32 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/1096928768' entity='client.admin' 2024-03-24T11:10:33.057 INFO:teuthology.orchestra.run.smithi062.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7P15b6kerwdAGK23zLWG6/zhUInHzJUYiQf9oQS3PmFYAMgu8ku/hdxcSo1Ekd0DIraph+boKnSlPILHo8ZuOwlYiX7OrWlvsiFUDzijK21wGhDdjY9dXxKuKRLaDMSpFUZDMW8BHCdP746jqg+hPfzK7krzOt9lpONIkFHJCDVSPiTSv5gGqLvGiS+PfFs0TnQlWnHZhq2hJRDcwwpfAp5LZ9vTwIoEL3YMei6RcgRPkZRugQ1hXy9fXjvpXqtlWfWO9RhN3uL+IjT8idQoOFF9PF4/dZsGs1VWR/TkaiXUMUPSyucWzE4PKTyazp/xEssRAsU+Dsk6aI4oHmYAQRWvkI6izEVW5froDVDEYE4ollgvo32TpOSPrUHOMxEXIUlh84Gqk5rop+zfQ5YNM7KzQAna/iuTg3b3SoF64cv9HlE3AHPHHMAPT/fvwttB5i2nsFALh3zRlYlwqeyN3Wvhkj8e4YKAvqe3P1Npo97110MKIYd3Eh5qJa2vAfQE= ceph-ebb89568-e9ce-11ee-95cd-87774f69a715 2024-03-24T11:10:33.221 DEBUG:teuthology.orchestra.run.smithi139:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7P15b6kerwdAGK23zLWG6/zhUInHzJUYiQf9oQS3PmFYAMgu8ku/hdxcSo1Ekd0DIraph+boKnSlPILHo8ZuOwlYiX7OrWlvsiFUDzijK21wGhDdjY9dXxKuKRLaDMSpFUZDMW8BHCdP746jqg+hPfzK7krzOt9lpONIkFHJCDVSPiTSv5gGqLvGiS+PfFs0TnQlWnHZhq2hJRDcwwpfAp5LZ9vTwIoEL3YMei6RcgRPkZRugQ1hXy9fXjvpXqtlWfWO9RhN3uL+IjT8idQoOFF9PF4/dZsGs1VWR/TkaiXUMUPSyucWzE4PKTyazp/xEssRAsU+Dsk6aI4oHmYAQRWvkI6izEVW5froDVDEYE4ollgvo32TpOSPrUHOMxEXIUlh84Gqk5rop+zfQ5YNM7KzQAna/iuTg3b3SoF64cv9HlE3AHPHHMAPT/fvwttB5i2nsFALh3zRlYlwqeyN3Wvhkj8e4YKAvqe3P1Npo97110MKIYd3Eh5qJa2vAfQE= ceph-ebb89568-e9ce-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-24T11:10:33.267 INFO:teuthology.orchestra.run.smithi139.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7P15b6kerwdAGK23zLWG6/zhUInHzJUYiQf9oQS3PmFYAMgu8ku/hdxcSo1Ekd0DIraph+boKnSlPILHo8ZuOwlYiX7OrWlvsiFUDzijK21wGhDdjY9dXxKuKRLaDMSpFUZDMW8BHCdP746jqg+hPfzK7krzOt9lpONIkFHJCDVSPiTSv5gGqLvGiS+PfFs0TnQlWnHZhq2hJRDcwwpfAp5LZ9vTwIoEL3YMei6RcgRPkZRugQ1hXy9fXjvpXqtlWfWO9RhN3uL+IjT8idQoOFF9PF4/dZsGs1VWR/TkaiXUMUPSyucWzE4PKTyazp/xEssRAsU+Dsk6aI4oHmYAQRWvkI6izEVW5froDVDEYE4ollgvo32TpOSPrUHOMxEXIUlh84Gqk5rop+zfQ5YNM7KzQAna/iuTg3b3SoF64cv9HlE3AHPHHMAPT/fvwttB5i2nsFALh3zRlYlwqeyN3Wvhkj8e4YKAvqe3P1Npo97110MKIYd3Eh5qJa2vAfQE= ceph-ebb89568-e9ce-11ee-95cd-87774f69a715 2024-03-24T11:10:33.282 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-24T11:10:33.726 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.62 - - [24/Mar/2024:11:10:33] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:10:34.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:33 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:10:34.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:33 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:10:34.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:33 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:10:34.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:33 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:10:34.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:33 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:10:34.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:33 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:10:34.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:33 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:10:37.254 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-24T11:10:37.255 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-24T11:10:37.791 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:37 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/1666289791' entity='client.admin' 2024-03-24T11:10:37.791 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:37 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:10:37.791 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:37 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:10:38.102 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:37 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.62 - - [24/Mar/2024:11:10:37] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:10:39.208 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:38 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:10:39.209 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:39 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.62 - - [24/Mar/2024:11:10:39] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:10:40.102 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi139 2024-03-24T11:10:40.103 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-03-24T11:10:40.103 DEBUG:teuthology.orchestra.run.smithi139:> dd of=/etc/ceph/ceph.conf 2024-03-24T11:10:40.122 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-03-24T11:10:40.122 DEBUG:teuthology.orchestra.run.smithi139:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-24T11:10:40.179 INFO:tasks.cephadm:Adding host smithi139 to orchestrator... 2024-03-24T11:10:40.179 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph orch host add smithi139 2024-03-24T11:10:40.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:40 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:10:40.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:40 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:10:40.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:40 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:10:40.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:40 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:10:40.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:40 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:10:40.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:40 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:10:40.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:40 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:10:40.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:40 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:10:41.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:41 smithi062 ceph-mon[24799]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:10:41.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:41 smithi062 ceph-mon[24799]: Updating smithi062:/etc/ceph/ceph.conf 2024-03-24T11:10:41.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:41 smithi062 ceph-mon[24799]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2024-03-24T11:10:42.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:42 smithi062 ceph-mon[24799]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi139", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:10:43.257 INFO:teuthology.orchestra.run.smithi062.stdout:Added host 'smithi139' with addr '172.21.15.139' 2024-03-24T11:10:43.815 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:43 smithi062 ceph-mon[24799]: Deploying cephadm binary to smithi139 2024-03-24T11:10:43.815 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:43 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:10:43.815 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:43 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:10:43.815 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:43 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:10:43.815 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:43 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:10:43.815 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:43 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:10:43.816 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph orch host ls --format=json 2024-03-24T11:10:45.201 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:44 smithi062 ceph-mon[24799]: Added host smithi139 2024-03-24T11:10:45.202 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:44 smithi062 ceph-mon[24799]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:10:45.202 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:44 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:10:45.202 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:44 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:10:45.202 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:44 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi139", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:10:45.202 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:44 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:10:45.202 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:44 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi139", "caps": []}]: dispatch 2024-03-24T11:10:45.202 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:44 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.agent.smithi139", "caps": []}]': finished 2024-03-24T11:10:45.202 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:44 smithi062 ceph-mon[24799]: Deploying daemon agent.smithi139 on smithi139 2024-03-24T11:10:45.202 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:44 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:10:45.202 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:44 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:10:45.202 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:44 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:10:45.202 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:44 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:10:45.722 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-24T11:10:45.723 INFO:teuthology.orchestra.run.smithi062.stdout:[{"addr": "172.21.15.62", "hostname": "smithi062", "labels": [], "status": ""}, {"addr": "172.21.15.139", "hostname": "smithi139", "labels": [], "status": ""}] 2024-03-24T11:10:45.835 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:45 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.139 - - [24/Mar/2024:11:10:45] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:10:46.103 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:45 smithi062 ceph-mon[24799]: Updating smithi139:/etc/ceph/ceph.conf 2024-03-24T11:10:46.103 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:45 smithi062 ceph-mon[24799]: Updating smithi139:/etc/ceph/ceph.client.admin.keyring 2024-03-24T11:10:46.103 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:45 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:10:46.103 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:45 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:10:46.103 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:45 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:10:46.103 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:45 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:10:46.103 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:45 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:10:46.103 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:45 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:10:46.103 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:45 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:10:46.329 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-24T11:10:46.329 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph osd crush tunables default 2024-03-24T11:10:46.354 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:46 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.139 - - [24/Mar/2024:11:10:46] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:10:47.371 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:47 smithi062 ceph-mon[24799]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-24T11:10:47.371 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:47 smithi062 ceph-mon[24799]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:10:47.371 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:47 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:10:47.371 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:47 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:10:47.976 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:10:47 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.139 - - [24/Mar/2024:11:10:47] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:10:48.790 INFO:teuthology.orchestra.run.smithi062.stderr:adjusted tunables profile to default 2024-03-24T11:10:48.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:48 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:10:48.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:48 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:10:48.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:48 smithi062 ceph-mon[24799]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:10:48.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:48 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/2338707341' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-24T11:10:49.455 INFO:tasks.cephadm:Adding mon.a on smithi062 2024-03-24T11:10:49.455 INFO:tasks.cephadm:Adding mon.c on smithi062 2024-03-24T11:10:49.455 INFO:tasks.cephadm:Adding mon.b on smithi139 2024-03-24T11:10:49.456 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph orch apply mon '3;smithi062:172.21.15.62=a;smithi062:[v2:172.21.15.62:3301,v1:172.21.15.62:6790]=c;smithi139:172.21.15.139=b' 2024-03-24T11:10:50.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:49 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/2338707341' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-24T11:10:50.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:49 smithi062 ceph-mon[24799]: osdmap e4: 0 total, 0 up, 0 in 2024-03-24T11:10:50.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:49 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:10:51.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:50 smithi062 ceph-mon[24799]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:10:51.299 INFO:teuthology.orchestra.run.smithi139.stdout:Scheduled mon update... 2024-03-24T11:10:51.812 DEBUG:teuthology.orchestra.run.smithi062:mon.c> sudo journalctl -f -n 0 -u ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mon.c.service 2024-03-24T11:10:51.814 DEBUG:teuthology.orchestra.run.smithi139:mon.b> sudo journalctl -f -n 0 -u ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mon.b.service 2024-03-24T11:10:51.816 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-24T11:10:51.817 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T11:10:52.641 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:52 smithi062 ceph-mon[24799]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi062:172.21.15.62=a;smithi062:[v2:172.21.15.62:3301,v1:172.21.15.62:6790]=c;smithi139:172.21.15.139=b", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:10:52.641 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:52 smithi062 ceph-mon[24799]: Saving service mon spec with placement smithi062:172.21.15.62=a;smithi062:[v2:172.21.15.62:3301,v1:172.21.15.62:6790]=c;smithi139:172.21.15.139=b;count:3 2024-03-24T11:10:52.641 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:52 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:10:52.642 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:52 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:10:52.642 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:52 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:10:52.642 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:52 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:10:52.642 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:52 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:10:52.642 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:52 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T11:10:52.642 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:52 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:10:52.642 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:52 smithi062 ceph-mon[24799]: Deploying daemon mon.c on smithi062 2024-03-24T11:10:52.642 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:52 smithi062 ceph-mon[24799]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:10:53.596 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-03-24T11:10:53.596 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":1,"fsid":"ebb89568-e9ce-11ee-95cd-87774f69a715","modified":"2024-03-24T11:09:30.820834Z","created":"2024-03-24T11:09:30.820834Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:3300","nonce":0},{"type":"v1","addr":"172.21.15.62:6789","nonce":0}]},"addr":"172.21.15.62:6789/0","public_addr":"172.21.15.62:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-24T11:10:53.598 INFO:teuthology.orchestra.run.smithi139.stderr:dumped monmap epoch 1 2024-03-24T11:10:53.889 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:53 smithi062 ceph-mon[24799]: from='client.? 172.21.15.139:0/1853317253' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T11:10:55.024 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:10:54 smithi062 ceph-mon[24799]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:10:55.024 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 podman[29626]: 2024-03-24 11:10:55.024310191 +0000 UTC m=+0.131414587 container create 39110a16ab6274f3f5d12019071044375633567274f6ac19b2c1cc44a62ccb2c (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-c, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, io.openshift.expose-services=, release=754, distribution-scope=public, CEPH_POINT_RELEASE=-17.2.0, GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True, vcs-type=git, vendor=Red Hat, Inc., maintainer=Guillaume Abrioux , vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, version=8, io.k8s.display-name=CentOS Stream 8, com.redhat.component=centos-stream-container, architecture=x86_64, name=centos-stream, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., GIT_CLEAN=True, GIT_BRANCH=HEAD, io.openshift.tags=base centos centos-stream, RELEASE=HEAD, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, build-date=2022-05-03T08:36:31.336870, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, io.buildah.version=1.19.8) 2024-03-24T11:10:55.173 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-24T11:10:55.173 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T11:10:55.579 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 podman[29626]: 2024-03-24 11:10:55.293986279 +0000 UTC m=+0.401090690 container init 39110a16ab6274f3f5d12019071044375633567274f6ac19b2c1cc44a62ccb2c (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-c, GIT_BRANCH=HEAD, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, io.openshift.expose-services=, ceph=True, architecture=x86_64, vcs-type=git, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, distribution-scope=public, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, io.openshift.tags=base centos centos-stream, CEPH_POINT_RELEASE=-17.2.0, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, build-date=2022-05-03T08:36:31.336870, vendor=Red Hat, Inc., com.redhat.license_terms=https://centos.org/legal/licensing-policy/, GIT_CLEAN=True, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.buildah.version=1.19.8, version=8, RELEASE=HEAD, com.redhat.component=centos-stream-container, maintainer=Guillaume Abrioux , io.k8s.display-name=CentOS Stream 8, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, GIT_REPO=https://github.com/ceph/ceph-container.git, name=centos-stream, release=754) 2024-03-24T11:10:55.579 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 podman[29626]: 2024-03-24 11:10:55.297059502 +0000 UTC m=+0.404163903 container start 39110a16ab6274f3f5d12019071044375633567274f6ac19b2c1cc44a62ccb2c (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-c, io.buildah.version=1.19.8, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, CEPH_POINT_RELEASE=-17.2.0, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, com.redhat.component=centos-stream-container, vendor=Red Hat, Inc., com.redhat.license_terms=https://centos.org/legal/licensing-policy/, name=centos-stream, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, io.openshift.tags=base centos centos-stream, GIT_REPO=https://github.com/ceph/ceph-container.git, vcs-type=git, io.k8s.display-name=CentOS Stream 8, release=754, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., RELEASE=HEAD, distribution-scope=public, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, version=8, build-date=2022-05-03T08:36:31.336870, architecture=x86_64, GIT_CLEAN=True, io.openshift.expose-services=, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com) 2024-03-24T11:10:55.579 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: set uid:gid to 167:167 (ceph:ceph) 2024-03-24T11:10:55.579 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable), process ceph-mon, pid 2 2024-03-24T11:10:55.579 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: pidfile_write: ignore empty --pid-file 2024-03-24T11:10:55.580 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: load: jerasure load: lrc 2024-03-24T11:10:55.580 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: RocksDB version: 6.15.5 2024-03-24T11:10:55.580 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-03-24T11:10:55.580 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Compile date Apr 18 2022 2024-03-24T11:10:55.580 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: DB SUMMARY 2024-03-24T11:10:55.580 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: DB Session ID: I5M64SBCFE4Q8JZ9I5V4 2024-03-24T11:10:55.580 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: CURRENT file: CURRENT 2024-03-24T11:10:55.580 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: IDENTITY file: IDENTITY 2024-03-24T11:10:55.580 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: MANIFEST file: MANIFEST-000003 size: 57 Bytes 2024-03-24T11:10:55.580 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-03-24T11:10:55.580 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-03-24T11:10:55.580 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.error_if_exists: 0 2024-03-24T11:10:55.581 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.create_if_missing: 0 2024-03-24T11:10:55.581 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.paranoid_checks: 1 2024-03-24T11:10:55.581 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-24T11:10:55.581 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.env: 0x560c670c6860 2024-03-24T11:10:55.581 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.fs: Posix File System 2024-03-24T11:10:55.581 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.info_log: 0x560c6950bdc0 2024-03-24T11:10:55.581 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.max_file_opening_threads: 16 2024-03-24T11:10:55.581 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.statistics: (nil) 2024-03-24T11:10:55.581 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.use_fsync: 0 2024-03-24T11:10:55.581 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.max_log_file_size: 0 2024-03-24T11:10:55.581 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-24T11:10:55.581 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-24T11:10:55.581 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.keep_log_file_num: 1000 2024-03-24T11:10:55.581 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.recycle_log_file_num: 0 2024-03-24T11:10:55.582 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.allow_fallocate: 1 2024-03-24T11:10:55.582 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.allow_mmap_reads: 0 2024-03-24T11:10:55.582 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.allow_mmap_writes: 0 2024-03-24T11:10:55.582 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.use_direct_reads: 0 2024-03-24T11:10:55.582 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-24T11:10:55.582 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.create_missing_column_families: 0 2024-03-24T11:10:55.582 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.db_log_dir: 2024-03-24T11:10:55.582 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-c/store.db 2024-03-24T11:10:55.582 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-24T11:10:55.582 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-24T11:10:55.582 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-24T11:10:55.582 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-24T11:10:55.582 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-24T11:10:55.583 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-24T11:10:55.584 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.advise_random_on_open: 1 2024-03-24T11:10:55.584 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.db_write_buffer_size: 0 2024-03-24T11:10:55.584 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.write_buffer_manager: 0x560c695fc2d0 2024-03-24T11:10:55.584 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-24T11:10:55.584 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-03-24T11:10:55.584 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-24T11:10:55.585 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-24T11:10:55.585 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.rate_limiter: (nil) 2024-03-24T11:10:55.585 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-24T11:10:55.585 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.wal_recovery_mode: 2 2024-03-24T11:10:55.585 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.enable_thread_tracking: 0 2024-03-24T11:10:55.585 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.enable_pipelined_write: 0 2024-03-24T11:10:55.585 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.unordered_write: 0 2024-03-24T11:10:55.585 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-24T11:10:55.585 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-24T11:10:55.585 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-24T11:10:55.585 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-24T11:10:55.586 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.row_cache: None 2024-03-24T11:10:55.586 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.wal_filter: None 2024-03-24T11:10:55.586 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-24T11:10:55.586 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.allow_ingest_behind: 0 2024-03-24T11:10:55.586 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.preserve_deletes: 0 2024-03-24T11:10:55.587 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.two_write_queues: 0 2024-03-24T11:10:55.587 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.manual_wal_flush: 0 2024-03-24T11:10:55.587 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.atomic_flush: 0 2024-03-24T11:10:55.587 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-24T11:10:55.587 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-24T11:10:55.587 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-24T11:10:55.587 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.log_readahead_size: 0 2024-03-24T11:10:55.587 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-24T11:10:55.587 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.best_efforts_recovery: 0 2024-03-24T11:10:55.587 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-24T11:10:55.587 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-24T11:10:55.587 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.allow_data_in_errors: 0 2024-03-24T11:10:55.587 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.db_host_id: __hostname__ 2024-03-24T11:10:55.587 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.max_background_jobs: 2 2024-03-24T11:10:55.588 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.max_background_compactions: -1 2024-03-24T11:10:55.588 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.max_subcompactions: 1 2024-03-24T11:10:55.588 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-24T11:10:55.588 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-24T11:10:55.588 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-24T11:10:55.588 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.max_total_wal_size: 0 2024-03-24T11:10:55.588 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-24T11:10:55.588 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-24T11:10:55.588 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-24T11:10:55.588 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-24T11:10:55.588 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.max_open_files: -1 2024-03-24T11:10:55.588 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.bytes_per_sync: 0 2024-03-24T11:10:55.588 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-24T11:10:55.588 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-24T11:10:55.588 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.compaction_readahead_size: 0 2024-03-24T11:10:55.589 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.max_background_flushes: -1 2024-03-24T11:10:55.589 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Compression algorithms supported: 2024-03-24T11:10:55.589 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-24T11:10:55.589 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: kZSTD supported: 0 2024-03-24T11:10:55.589 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: kXpressCompression supported: 0 2024-03-24T11:10:55.589 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: kLZ4HCCompression supported: 1 2024-03-24T11:10:55.589 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: kLZ4Compression supported: 1 2024-03-24T11:10:55.589 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: kBZip2Compression supported: 0 2024-03-24T11:10:55.589 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: kZlibCompression supported: 1 2024-03-24T11:10:55.589 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: kSnappyCompression supported: 1 2024-03-24T11:10:55.589 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-24T11:10:55.589 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: [db/version_set.cc:4725] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000003 2024-03-24T11:10:55.589 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: [db/column_family.cc:597] --------------- Options for column family [default]: 2024-03-24T11:10:55.589 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-24T11:10:55.589 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.merge_operator: 2024-03-24T11:10:55.590 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.compaction_filter: None 2024-03-24T11:10:55.590 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.compaction_filter_factory: None 2024-03-24T11:10:55.590 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.sst_partitioner_factory: None 2024-03-24T11:10:55.590 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-24T11:10:55.590 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-24T11:10:55.590 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560c694d7d00) 2024-03-24T11:10:55.590 INFO:journalctl@ceph.mon.c.smithi062.stdout: cache_index_and_filter_blocks: 1 2024-03-24T11:10:55.590 INFO:journalctl@ceph.mon.c.smithi062.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-24T11:10:55.590 INFO:journalctl@ceph.mon.c.smithi062.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-24T11:10:55.590 INFO:journalctl@ceph.mon.c.smithi062.stdout: pin_top_level_index_and_filter: 1 2024-03-24T11:10:55.590 INFO:journalctl@ceph.mon.c.smithi062.stdout: index_type: 0 2024-03-24T11:10:55.590 INFO:journalctl@ceph.mon.c.smithi062.stdout: data_block_index_type: 0 2024-03-24T11:10:55.590 INFO:journalctl@ceph.mon.c.smithi062.stdout: index_shortening: 1 2024-03-24T11:10:55.590 INFO:journalctl@ceph.mon.c.smithi062.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-24T11:10:55.591 INFO:journalctl@ceph.mon.c.smithi062.stdout: hash_index_allow_collision: 1 2024-03-24T11:10:55.591 INFO:journalctl@ceph.mon.c.smithi062.stdout: checksum: 1 2024-03-24T11:10:55.591 INFO:journalctl@ceph.mon.c.smithi062.stdout: no_block_cache: 0 2024-03-24T11:10:55.591 INFO:journalctl@ceph.mon.c.smithi062.stdout: block_cache: 0x560c69542170 2024-03-24T11:10:55.591 INFO:journalctl@ceph.mon.c.smithi062.stdout: block_cache_name: BinnedLRUCache 2024-03-24T11:10:55.591 INFO:journalctl@ceph.mon.c.smithi062.stdout: block_cache_options: 2024-03-24T11:10:55.591 INFO:journalctl@ceph.mon.c.smithi062.stdout: capacity : 536870912 2024-03-24T11:10:55.591 INFO:journalctl@ceph.mon.c.smithi062.stdout: num_shard_bits : 4 2024-03-24T11:10:55.591 INFO:journalctl@ceph.mon.c.smithi062.stdout: strict_capacity_limit : 0 2024-03-24T11:10:55.591 INFO:journalctl@ceph.mon.c.smithi062.stdout: high_pri_pool_ratio: 0.000 2024-03-24T11:10:55.591 INFO:journalctl@ceph.mon.c.smithi062.stdout: block_cache_compressed: (nil) 2024-03-24T11:10:55.591 INFO:journalctl@ceph.mon.c.smithi062.stdout: persistent_cache: (nil) 2024-03-24T11:10:55.591 INFO:journalctl@ceph.mon.c.smithi062.stdout: block_size: 4096 2024-03-24T11:10:55.591 INFO:journalctl@ceph.mon.c.smithi062.stdout: block_size_deviation: 10 2024-03-24T11:10:55.591 INFO:journalctl@ceph.mon.c.smithi062.stdout: block_restart_interval: 16 2024-03-24T11:10:55.592 INFO:journalctl@ceph.mon.c.smithi062.stdout: index_block_restart_interval: 1 2024-03-24T11:10:55.592 INFO:journalctl@ceph.mon.c.smithi062.stdout: metadata_block_size: 4096 2024-03-24T11:10:55.592 INFO:journalctl@ceph.mon.c.smithi062.stdout: partition_filters: 0 2024-03-24T11:10:55.592 INFO:journalctl@ceph.mon.c.smithi062.stdout: use_delta_encoding: 1 2024-03-24T11:10:55.592 INFO:journalctl@ceph.mon.c.smithi062.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-03-24T11:10:55.592 INFO:journalctl@ceph.mon.c.smithi062.stdout: whole_key_filtering: 1 2024-03-24T11:10:55.592 INFO:journalctl@ceph.mon.c.smithi062.stdout: verify_compression: 0 2024-03-24T11:10:55.592 INFO:journalctl@ceph.mon.c.smithi062.stdout: read_amp_bytes_per_bit: 0 2024-03-24T11:10:55.592 INFO:journalctl@ceph.mon.c.smithi062.stdout: format_version: 4 2024-03-24T11:10:55.592 INFO:journalctl@ceph.mon.c.smithi062.stdout: enable_index_compression: 1 2024-03-24T11:10:55.592 INFO:journalctl@ceph.mon.c.smithi062.stdout: block_align: 0 2024-03-24T11:10:55.593 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.write_buffer_size: 33554432 2024-03-24T11:10:55.593 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.max_write_buffer_number: 2 2024-03-24T11:10:55.593 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.compression: NoCompression 2024-03-24T11:10:55.593 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.bottommost_compression: Disabled 2024-03-24T11:10:55.593 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.prefix_extractor: nullptr 2024-03-24T11:10:55.593 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-24T11:10:55.593 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.num_levels: 7 2024-03-24T11:10:55.593 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-24T11:10:55.593 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-24T11:10:55.593 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-24T11:10:55.593 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-24T11:10:55.593 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-24T11:10:55.593 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-24T11:10:55.593 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-24T11:10:55.594 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-24T11:10:55.594 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-24T11:10:55.594 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-24T11:10:55.594 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-24T11:10:55.594 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.compression_opts.level: 32767 2024-03-24T11:10:55.594 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.compression_opts.strategy: 0 2024-03-24T11:10:55.594 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-24T11:10:55.594 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-24T11:10:55.594 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-24T11:10:55.594 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.compression_opts.enabled: false 2024-03-24T11:10:55.594 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-24T11:10:55.594 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-24T11:10:55.594 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-24T11:10:55.594 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.target_file_size_base: 67108864 2024-03-24T11:10:55.595 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-24T11:10:55.595 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-24T11:10:55.595 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-24T11:10:55.595 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-24T11:10:55.595 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-24T11:10:55.595 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-24T11:10:55.595 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-24T11:10:55.595 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-24T11:10:55.595 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-24T11:10:55.595 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-24T11:10:55.595 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-24T11:10:55.595 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-24T11:10:55.595 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-24T11:10:55.595 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.arena_block_size: 4194304 2024-03-24T11:10:55.595 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-24T11:10:55.596 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-24T11:10:55.596 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-03-24T11:10:55.596 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.disable_auto_compactions: 0 2024-03-24T11:10:55.596 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-24T11:10:55.596 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-24T11:10:55.596 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-24T11:10:55.596 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-24T11:10:55.596 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-24T11:10:55.596 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-24T11:10:55.596 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-24T11:10:55.597 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-24T11:10:55.597 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-24T11:10:55.597 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-24T11:10:55.598 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.table_properties_collectors: 2024-03-24T11:10:55.598 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.inplace_update_support: 0 2024-03-24T11:10:55.598 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-24T11:10:55.598 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-24T11:10:55.598 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-24T11:10:55.598 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-24T11:10:55.598 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.bloom_locality: 0 2024-03-24T11:10:55.598 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.max_successive_merges: 0 2024-03-24T11:10:55.598 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-24T11:10:55.598 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.paranoid_file_checks: 0 2024-03-24T11:10:55.598 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.force_consistency_checks: 1 2024-03-24T11:10:55.598 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.report_bg_io_stats: 0 2024-03-24T11:10:55.599 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.ttl: 2592000 2024-03-24T11:10:55.599 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-24T11:10:55.599 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.enable_blob_files: false 2024-03-24T11:10:55.599 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.min_blob_size: 0 2024-03-24T11:10:55.599 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.blob_file_size: 268435456 2024-03-24T11:10:55.599 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-24T11:10:55.599 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-24T11:10:55.599 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-24T11:10:55.599 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: [db/version_set.cc:4773] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000003 succeeded,manifest_file_number is 3, next_file_number is 5, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-24T11:10:55.599 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: [db/version_set.cc:4782] Column family [default] (ID 0), log number is 0 2024-03-24T11:10:55.599 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: [db/version_set.cc:4083] Creating manifest 7 2024-03-24T11:10:55.599 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711278655322659, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-24T11:10:55.599 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: [db/db_impl/db_impl_open.cc:847] Recovering log #4 mode 2 2024-03-24T11:10:55.599 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: [table/block_based/filter_policy.cc:996] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-03-24T11:10:55.600 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711278655322981, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1545, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1711278655, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "b6db2122-ff52-4d85-81c5-579d4a8837d2", "db_session_id": "I5M64SBCFE4Q8JZ9I5V4"}} 2024-03-24T11:10:55.600 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: [db/version_set.cc:4083] Creating manifest 9 2024-03-24T11:10:55.600 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711278655323328, "job": 1, "event": "recovery_finished"} 2024-03-24T11:10:55.600 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: [file/delete_scheduler.cc:73] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-24T11:10:55.600 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: [db/db_impl/db_impl_open.cc:1701] SstFileManager instance 0x560c69528a80 2024-03-24T11:10:55.600 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: DB pointer 0x560c69538000 2024-03-24T11:10:55.600 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: [db/db_impl/db_impl.cc:902] ------- DUMPING STATS ------- 2024-03-24T11:10:55.600 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-03-24T11:10:55.600 INFO:journalctl@ceph.mon.c.smithi062.stdout: ** DB Stats ** 2024-03-24T11:10:55.600 INFO:journalctl@ceph.mon.c.smithi062.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-24T11:10:55.600 INFO:journalctl@ceph.mon.c.smithi062.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-24T11:10:55.600 INFO:journalctl@ceph.mon.c.smithi062.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-24T11:10:55.600 INFO:journalctl@ceph.mon.c.smithi062.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-24T11:10:55.600 INFO:journalctl@ceph.mon.c.smithi062.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-24T11:10:55.601 INFO:journalctl@ceph.mon.c.smithi062.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-03-24T11:10:55.601 INFO:journalctl@ceph.mon.c.smithi062.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-24T11:10:55.601 INFO:journalctl@ceph.mon.c.smithi062.stdout: 2024-03-24T11:10:55.601 INFO:journalctl@ceph.mon.c.smithi062.stdout: ** Compaction Stats [default] ** 2024-03-24T11:10:55.601 INFO:journalctl@ceph.mon.c.smithi062.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-24T11:10:55.601 INFO:journalctl@ceph.mon.c.smithi062.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-24T11:10:55.601 INFO:journalctl@ceph.mon.c.smithi062.stdout: L0 1/0 1.51 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.7 0.00 0.00 1 0.000 0 0 2024-03-24T11:10:55.601 INFO:journalctl@ceph.mon.c.smithi062.stdout: Sum 1/0 1.51 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.7 0.00 0.00 1 0.000 0 0 2024-03-24T11:10:55.601 INFO:journalctl@ceph.mon.c.smithi062.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.7 0.00 0.00 1 0.000 0 0 2024-03-24T11:10:55.601 INFO:journalctl@ceph.mon.c.smithi062.stdout: 2024-03-24T11:10:55.601 INFO:journalctl@ceph.mon.c.smithi062.stdout: ** Compaction Stats [default] ** 2024-03-24T11:10:55.601 INFO:journalctl@ceph.mon.c.smithi062.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-24T11:10:55.601 INFO:journalctl@ceph.mon.c.smithi062.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-24T11:10:55.601 INFO:journalctl@ceph.mon.c.smithi062.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.7 0.00 0.00 1 0.000 0 0 2024-03-24T11:10:55.602 INFO:journalctl@ceph.mon.c.smithi062.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-24T11:10:55.602 INFO:journalctl@ceph.mon.c.smithi062.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-24T11:10:55.602 INFO:journalctl@ceph.mon.c.smithi062.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-24T11:10:55.602 INFO:journalctl@ceph.mon.c.smithi062.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-24T11:10:55.602 INFO:journalctl@ceph.mon.c.smithi062.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-24T11:10:55.602 INFO:journalctl@ceph.mon.c.smithi062.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-24T11:10:55.602 INFO:journalctl@ceph.mon.c.smithi062.stdout: Cumulative compaction: 0.00 GB write, 0.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-24T11:10:55.602 INFO:journalctl@ceph.mon.c.smithi062.stdout: Interval compaction: 0.00 GB write, 0.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-24T11:10:55.602 INFO:journalctl@ceph.mon.c.smithi062.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-24T11:10:55.603 INFO:journalctl@ceph.mon.c.smithi062.stdout: 2024-03-24T11:10:55.603 INFO:journalctl@ceph.mon.c.smithi062.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-24T11:10:55.603 INFO:journalctl@ceph.mon.c.smithi062.stdout: 2024-03-24T11:10:55.603 INFO:journalctl@ceph.mon.c.smithi062.stdout: ** Compaction Stats [default] ** 2024-03-24T11:10:55.603 INFO:journalctl@ceph.mon.c.smithi062.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-24T11:10:55.603 INFO:journalctl@ceph.mon.c.smithi062.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-24T11:10:55.603 INFO:journalctl@ceph.mon.c.smithi062.stdout: L0 1/0 1.51 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.7 0.00 0.00 1 0.000 0 0 2024-03-24T11:10:55.603 INFO:journalctl@ceph.mon.c.smithi062.stdout: Sum 1/0 1.51 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.7 0.00 0.00 1 0.000 0 0 2024-03-24T11:10:55.603 INFO:journalctl@ceph.mon.c.smithi062.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-03-24T11:10:55.603 INFO:journalctl@ceph.mon.c.smithi062.stdout: 2024-03-24T11:10:55.603 INFO:journalctl@ceph.mon.c.smithi062.stdout: ** Compaction Stats [default] ** 2024-03-24T11:10:55.603 INFO:journalctl@ceph.mon.c.smithi062.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-24T11:10:55.603 INFO:journalctl@ceph.mon.c.smithi062.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-24T11:10:55.604 INFO:journalctl@ceph.mon.c.smithi062.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.7 0.00 0.00 1 0.000 0 0 2024-03-24T11:10:55.604 INFO:journalctl@ceph.mon.c.smithi062.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-24T11:10:55.604 INFO:journalctl@ceph.mon.c.smithi062.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-24T11:10:55.604 INFO:journalctl@ceph.mon.c.smithi062.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-24T11:10:55.604 INFO:journalctl@ceph.mon.c.smithi062.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-24T11:10:55.604 INFO:journalctl@ceph.mon.c.smithi062.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-24T11:10:55.604 INFO:journalctl@ceph.mon.c.smithi062.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-24T11:10:55.604 INFO:journalctl@ceph.mon.c.smithi062.stdout: Cumulative compaction: 0.00 GB write, 0.57 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-24T11:10:55.604 INFO:journalctl@ceph.mon.c.smithi062.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-24T11:10:55.604 INFO:journalctl@ceph.mon.c.smithi062.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-24T11:10:55.604 INFO:journalctl@ceph.mon.c.smithi062.stdout: 2024-03-24T11:10:55.604 INFO:journalctl@ceph.mon.c.smithi062.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-24T11:10:55.604 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-03-24T11:10:55.604 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: using public_addrv [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] 2024-03-24T11:10:55.605 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: starting mon.c rank -1 at public addrs [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] at bind addrs [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] mon_data /var/lib/ceph/mon/ceph-c fsid ebb89568-e9ce-11ee-95cd-87774f69a715 2024-03-24T11:10:55.605 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: mon.c@-1(???) e0 preinit fsid ebb89568-e9ce-11ee-95cd-87774f69a715 2024-03-24T11:10:55.605 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: mon.c@-1(synchronizing).mds e1 new map 2024-03-24T11:10:55.605 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: mon.c@-1(synchronizing).mds e1 print_map 2024-03-24T11:10:55.605 INFO:journalctl@ceph.mon.c.smithi062.stdout: e1 2024-03-24T11:10:55.605 INFO:journalctl@ceph.mon.c.smithi062.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T11:10:55.605 INFO:journalctl@ceph.mon.c.smithi062.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T11:10:55.605 INFO:journalctl@ceph.mon.c.smithi062.stdout: legacy client fscid: -1 2024-03-24T11:10:55.605 INFO:journalctl@ceph.mon.c.smithi062.stdout: 2024-03-24T11:10:55.606 INFO:journalctl@ceph.mon.c.smithi062.stdout: No filesystems configured 2024-03-24T11:10:55.606 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-24T11:10:55.606 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-24T11:10:55.606 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-24T11:10:55.606 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-24T11:10:55.606 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-24T11:10:55.606 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-24T11:10:55.606 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-24T11:10:55.606 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-24T11:10:55.606 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-24T11:10:55.606 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-24T11:10:55.606 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: Added host smithi139 2024-03-24T11:10:55.606 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:10:55.607 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:10:55.607 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:10:55.607 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi139", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:10:55.607 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:10:55.607 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi139", "caps": []}]: dispatch 2024-03-24T11:10:55.607 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.agent.smithi139", "caps": []}]': finished 2024-03-24T11:10:55.607 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: Deploying daemon agent.smithi139 on smithi139 2024-03-24T11:10:55.607 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:10:55.607 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:10:55.607 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:10:55.608 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:10:55.608 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: Updating smithi139:/etc/ceph/ceph.conf 2024-03-24T11:10:55.608 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: Updating smithi139:/etc/ceph/ceph.client.admin.keyring 2024-03-24T11:10:55.608 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:10:55.608 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:10:55.608 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:10:55.608 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:10:55.608 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:10:55.608 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:10:55.608 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:10:55.609 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-24T11:10:55.609 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:10:55.609 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:10:55.609 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:10:55.609 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:10:55.609 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:10:55.609 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:10:55.609 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/2338707341' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-24T11:10:55.609 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/2338707341' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-24T11:10:55.609 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: osdmap e4: 0 total, 0 up, 0 in 2024-03-24T11:10:55.609 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:10:55.609 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:10:55.609 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi062:172.21.15.62=a;smithi062:[v2:172.21.15.62:3301,v1:172.21.15.62:6790]=c;smithi139:172.21.15.139=b", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:10:55.609 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: Saving service mon spec with placement smithi062:172.21.15.62=a;smithi062:[v2:172.21.15.62:3301,v1:172.21.15.62:6790]=c;smithi139:172.21.15.139=b;count:3 2024-03-24T11:10:55.609 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:10:55.610 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:10:55.610 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:10:55.610 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:10:55.610 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:10:55.610 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T11:10:55.610 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:10:55.610 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: Deploying daemon mon.c on smithi062 2024-03-24T11:10:55.610 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:10:55.610 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: from='client.? 172.21.15.139:0/1853317253' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T11:10:55.610 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:10:55.610 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: mon.c@-1(synchronizing).paxosservice(auth 1..5) refresh upgraded, format 0 -> 3 2024-03-24T11:10:55.610 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 ceph-mon[29666]: mon.c@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete,mon_cluster_log_to_file 2024-03-24T11:10:55.610 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 bash[29626]: 39110a16ab6274f3f5d12019071044375633567274f6ac19b2c1cc44a62ccb2c 2024-03-24T11:10:55.610 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:10:55 smithi062 systemd[1]: Started Ceph mon.c for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:11:00.727 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:11:00 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.62 - - [24/Mar/2024:11:11:00] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:11:00.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:00 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T11:11:00.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:00 smithi062 ceph-mon[24799]: mon.a calling monitor election 2024-03-24T11:11:00.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:00 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:11:00.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:00 smithi062 ceph-mon[24799]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:00.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:00 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:11:00.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:00 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:11:00.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:00 smithi062 ceph-mon[24799]: mon.c calling monitor election 2024-03-24T11:11:00.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:00 smithi062 ceph-mon[24799]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:00.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:00 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:11:00.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:00 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:11:00.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:00 smithi062 ceph-mon[24799]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:00.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:00 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:11:00.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:00 smithi062 ceph-mon[24799]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-24T11:11:00.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:00 smithi062 ceph-mon[24799]: monmap e2: 2 mons at {a=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0],c=[v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0]} 2024-03-24T11:11:00.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:00 smithi062 ceph-mon[24799]: fsmap 2024-03-24T11:11:00.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:00 smithi062 ceph-mon[24799]: osdmap e4: 0 total, 0 up, 0 in 2024-03-24T11:11:00.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:00 smithi062 ceph-mon[24799]: mgrmap e13: y(active, since 37s) 2024-03-24T11:11:00.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:00 smithi062 ceph-mon[24799]: overall HEALTH_OK 2024-03-24T11:11:00.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:00 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:00.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:00 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T11:11:00.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:00 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:00.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:00 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:00.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:00 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:00.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:00 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T11:11:00.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:00 smithi062 ceph-mon[29666]: mon.a calling monitor election 2024-03-24T11:11:00.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:00 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:11:00.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:00 smithi062 ceph-mon[29666]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:00.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:00 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:11:00.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:00 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:11:00.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:00 smithi062 ceph-mon[29666]: mon.c calling monitor election 2024-03-24T11:11:00.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:00 smithi062 ceph-mon[29666]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:00.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:00 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:11:00.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:00 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:11:00.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:00 smithi062 ceph-mon[29666]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:00.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:00 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:11:00.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:00 smithi062 ceph-mon[29666]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-24T11:11:00.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:00 smithi062 ceph-mon[29666]: monmap e2: 2 mons at {a=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0],c=[v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0]} 2024-03-24T11:11:00.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:00 smithi062 ceph-mon[29666]: fsmap 2024-03-24T11:11:00.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:00 smithi062 ceph-mon[29666]: osdmap e4: 0 total, 0 up, 0 in 2024-03-24T11:11:00.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:00 smithi062 ceph-mon[29666]: mgrmap e13: y(active, since 37s) 2024-03-24T11:11:00.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:00 smithi062 ceph-mon[29666]: overall HEALTH_OK 2024-03-24T11:11:00.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:00 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:00.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:00 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T11:11:00.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:00 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:00.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:00 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:00.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:00 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:01.362 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-03-24T11:11:01.363 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":2,"fsid":"ebb89568-e9ce-11ee-95cd-87774f69a715","modified":"2024-03-24T11:10:55.341571Z","created":"2024-03-24T11:09:30.820834Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:3300","nonce":0},{"type":"v1","addr":"172.21.15.62:6789","nonce":0}]},"addr":"172.21.15.62:6789/0","public_addr":"172.21.15.62:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:3301","nonce":0},{"type":"v1","addr":"172.21.15.62:6790","nonce":0}]},"addr":"172.21.15.62:6790/0","public_addr":"172.21.15.62:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-24T11:11:01.365 INFO:teuthology.orchestra.run.smithi139.stderr:dumped monmap epoch 2 2024-03-24T11:11:01.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:01 smithi062 ceph-mon[24799]: Deploying daemon mon.b on smithi139 2024-03-24T11:11:01.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:01 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:11:01.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:01 smithi062 ceph-mon[24799]: from='client.? 172.21.15.139:0/4193236733' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T11:11:01.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:01 smithi062 ceph-mon[29666]: Deploying daemon mon.b on smithi139 2024-03-24T11:11:01.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:01 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:11:01.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:01 smithi062 ceph-mon[29666]: from='client.? 172.21.15.139:0/4193236733' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T11:11:02.726 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:02 smithi062 ceph-mon[24799]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:02.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:02 smithi062 ceph-mon[29666]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:03.986 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-24T11:11:03.986 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T11:11:04.677 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-24T11:11:04.677 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-24T11:11:04.678 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-24T11:11:04.678 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-24T11:11:04.678 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-24T11:11:04.678 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-24T11:11:04.934 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-24T11:11:04.934 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-24T11:11:04.934 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-24T11:11:04.934 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-24T11:11:04.934 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: Added host smithi139 2024-03-24T11:11:04.935 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:04.935 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:04.935 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:04.935 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi139", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:11:04.935 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:04.935 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi139", "caps": []}]: dispatch 2024-03-24T11:11:04.935 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.agent.smithi139", "caps": []}]': finished 2024-03-24T11:11:04.935 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: Deploying daemon agent.smithi139 on smithi139 2024-03-24T11:11:04.935 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:04.935 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:11:04.935 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:04.935 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:11:04.935 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: Updating smithi139:/etc/ceph/ceph.conf 2024-03-24T11:11:04.936 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: Updating smithi139:/etc/ceph/ceph.client.admin.keyring 2024-03-24T11:11:04.936 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:04.936 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:04.936 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:11:04.936 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:04.936 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:11:04.936 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:04.936 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:04.936 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-24T11:11:04.936 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:04.936 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:04.936 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:04.936 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:04.936 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:04.937 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:04.937 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/2338707341' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-24T11:11:04.937 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/2338707341' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-24T11:11:04.937 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: osdmap e4: 0 total, 0 up, 0 in 2024-03-24T11:11:04.937 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:04.937 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:04.937 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi062:172.21.15.62=a;smithi062:[v2:172.21.15.62:3301,v1:172.21.15.62:6790]=c;smithi139:172.21.15.139=b", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:11:04.937 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: Saving service mon spec with placement smithi062:172.21.15.62=a;smithi062:[v2:172.21.15.62:3301,v1:172.21.15.62:6790]=c;smithi139:172.21.15.139=b;count:3 2024-03-24T11:11:04.937 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:04.937 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:11:04.937 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:04.937 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:11:04.937 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:04.937 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T11:11:04.937 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:04.938 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: Deploying daemon mon.c on smithi062 2024-03-24T11:11:04.938 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:04.938 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: from='client.? 172.21.15.139:0/1853317253' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T11:11:04.938 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:04.938 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T11:11:04.938 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: mon.a calling monitor election 2024-03-24T11:11:04.938 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:11:04.938 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:04.938 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:11:04.938 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:11:04.938 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: mon.c calling monitor election 2024-03-24T11:11:04.938 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:04.938 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:11:04.938 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:11:04.939 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:04.939 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:11:04.939 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-24T11:11:04.939 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: monmap e2: 2 mons at {a=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0],c=[v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0]} 2024-03-24T11:11:04.939 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: fsmap 2024-03-24T11:11:04.939 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: osdmap e4: 0 total, 0 up, 0 in 2024-03-24T11:11:04.939 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: mgrmap e13: y(active, since 37s) 2024-03-24T11:11:04.939 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: overall HEALTH_OK 2024-03-24T11:11:04.939 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:04.939 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T11:11:04.939 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:04.939 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:04.939 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:04.939 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: Deploying daemon mon.b on smithi139 2024-03-24T11:11:04.939 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:11:04.940 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: from='client.? 172.21.15.139:0/4193236733' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T11:11:04.940 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:04.940 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: mon.b@-1(synchronizing).paxosservice(auth 1..5) refresh upgraded, format 0 -> 3 2024-03-24T11:11:04.940 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 ceph-mon[25719]: mon.b@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete,mon_cluster_log_to_file 2024-03-24T11:11:04.940 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 bash[25666]: 8945cdb4e6aa0e97119eb73f8d53fb090369f42a3fd04dbbb49c2472a4879024 2024-03-24T11:11:04.940 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:04 smithi139 systemd[1]: Started Ceph mon.b for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:11:09.977 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.139 - - [24/Mar/2024:11:11:09] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:11:09.977 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.139 - - [24/Mar/2024:11:11:09] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:11:09.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T11:11:09.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[29666]: mon.a calling monitor election 2024-03-24T11:11:09.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[29666]: mon.c calling monitor election 2024-03-24T11:11:09.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T11:11:09.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:11:09.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T11:11:09.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[29666]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:09.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T11:11:09.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[29666]: mon.b calling monitor election 2024-03-24T11:11:09.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T11:11:09.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[29666]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:09.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T11:11:09.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T11:11:09.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[29666]: mon.a calling monitor election 2024-03-24T11:11:09.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[29666]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-24T11:11:09.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[29666]: monmap e3: 3 mons at {a=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0],b=[v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0],c=[v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0]} 2024-03-24T11:11:09.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[29666]: fsmap 2024-03-24T11:11:09.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[29666]: osdmap e4: 0 total, 0 up, 0 in 2024-03-24T11:11:09.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[29666]: mgrmap e13: y(active, since 46s) 2024-03-24T11:11:09.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[29666]: overall HEALTH_OK 2024-03-24T11:11:09.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:09.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:09.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:11:09.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:09.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:09.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:11:09.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T11:11:09.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[24799]: mon.a calling monitor election 2024-03-24T11:11:09.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[24799]: mon.c calling monitor election 2024-03-24T11:11:09.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T11:11:09.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:11:09.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T11:11:09.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[24799]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:09.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T11:11:09.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[24799]: mon.b calling monitor election 2024-03-24T11:11:09.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T11:11:09.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[24799]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:09.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T11:11:09.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T11:11:09.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[24799]: mon.a calling monitor election 2024-03-24T11:11:09.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[24799]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-24T11:11:09.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[24799]: monmap e3: 3 mons at {a=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0],b=[v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0],c=[v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0]} 2024-03-24T11:11:09.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[24799]: fsmap 2024-03-24T11:11:09.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[24799]: osdmap e4: 0 total, 0 up, 0 in 2024-03-24T11:11:09.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[24799]: mgrmap e13: y(active, since 46s) 2024-03-24T11:11:09.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[24799]: overall HEALTH_OK 2024-03-24T11:11:09.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:09.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:09.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:11:09.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:09.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:09.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:09 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:11:09.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:09 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T11:11:09.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:09 smithi139 ceph-mon[25719]: mon.a calling monitor election 2024-03-24T11:11:09.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:09 smithi139 ceph-mon[25719]: mon.c calling monitor election 2024-03-24T11:11:09.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:09 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T11:11:09.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:09 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:11:09.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:09 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T11:11:09.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:09 smithi139 ceph-mon[25719]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:09.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:09 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T11:11:09.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:09 smithi139 ceph-mon[25719]: mon.b calling monitor election 2024-03-24T11:11:09.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:09 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T11:11:10.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:09 smithi139 ceph-mon[25719]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:10.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:09 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T11:11:10.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:09 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T11:11:10.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:09 smithi139 ceph-mon[25719]: mon.a calling monitor election 2024-03-24T11:11:10.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:09 smithi139 ceph-mon[25719]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-24T11:11:10.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:09 smithi139 ceph-mon[25719]: monmap e3: 3 mons at {a=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0],b=[v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0],c=[v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0]} 2024-03-24T11:11:10.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:09 smithi139 ceph-mon[25719]: fsmap 2024-03-24T11:11:10.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:09 smithi139 ceph-mon[25719]: osdmap e4: 0 total, 0 up, 0 in 2024-03-24T11:11:10.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:09 smithi139 ceph-mon[25719]: mgrmap e13: y(active, since 46s) 2024-03-24T11:11:10.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:09 smithi139 ceph-mon[25719]: overall HEALTH_OK 2024-03-24T11:11:10.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:09 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:10.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:09 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:10.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:09 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:11:10.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:09 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:10.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:09 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:10.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:09 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:11:10.300 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-03-24T11:11:10.300 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":3,"fsid":"ebb89568-e9ce-11ee-95cd-87774f69a715","modified":"2024-03-24T11:11:04.686970Z","created":"2024-03-24T11:09:30.820834Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:3300","nonce":0},{"type":"v1","addr":"172.21.15.62:6789","nonce":0}]},"addr":"172.21.15.62:6789/0","public_addr":"172.21.15.62:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:3301","nonce":0},{"type":"v1","addr":"172.21.15.62:6790","nonce":0}]},"addr":"172.21.15.62:6790/0","public_addr":"172.21.15.62:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:3300","nonce":0},{"type":"v1","addr":"172.21.15.139:6789","nonce":0}]},"addr":"172.21.15.139:6789/0","public_addr":"172.21.15.139:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-03-24T11:11:10.301 INFO:teuthology.orchestra.run.smithi139.stderr:dumped monmap epoch 3 2024-03-24T11:11:10.891 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:10 smithi139 ceph-mon[25719]: Updating smithi062:/etc/ceph/ceph.conf 2024-03-24T11:11:10.892 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:10 smithi139 ceph-mon[25719]: Updating smithi139:/etc/ceph/ceph.conf 2024-03-24T11:11:10.892 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:10 smithi139 ceph-mon[25719]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:10.892 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:10 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:10.892 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:10 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:10.892 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:10 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:10.892 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:10 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:10.892 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:10 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:10.892 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:10 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:11:10.892 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:10 smithi139 ceph-mon[25719]: Reconfiguring mon.a (unknown last config time)... 2024-03-24T11:11:10.892 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:10 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T11:11:10.892 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:10 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T11:11:10.893 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:10 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:10.893 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:10 smithi139 ceph-mon[25719]: Reconfiguring daemon mon.a on smithi062 2024-03-24T11:11:10.893 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:10 smithi139 ceph-mon[25719]: from='client.? 172.21.15.139:0/468480649' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T11:11:10.893 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:10 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T11:11:10.914 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-24T11:11:10.914 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph config generate-minimal-conf 2024-03-24T11:11:10.935 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:10 smithi062 ceph-mon[29666]: Updating smithi062:/etc/ceph/ceph.conf 2024-03-24T11:11:10.936 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:10 smithi062 ceph-mon[29666]: Updating smithi139:/etc/ceph/ceph.conf 2024-03-24T11:11:10.936 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:10 smithi062 ceph-mon[29666]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:10.936 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:10 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:10.936 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:10 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:10.936 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:10 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:10.936 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:10 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:10.936 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:10 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:10.936 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:10 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:11:10.936 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:10 smithi062 ceph-mon[29666]: Reconfiguring mon.a (unknown last config time)... 2024-03-24T11:11:10.936 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:10 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T11:11:10.936 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:10 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T11:11:10.936 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:10 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:10.936 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:10 smithi062 ceph-mon[29666]: Reconfiguring daemon mon.a on smithi062 2024-03-24T11:11:10.937 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:10 smithi062 ceph-mon[29666]: from='client.? 172.21.15.139:0/468480649' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T11:11:10.937 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:10 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T11:11:10.937 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:10 smithi062 ceph-mon[24799]: Updating smithi062:/etc/ceph/ceph.conf 2024-03-24T11:11:10.937 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:10 smithi062 ceph-mon[24799]: Updating smithi139:/etc/ceph/ceph.conf 2024-03-24T11:11:10.937 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:10 smithi062 ceph-mon[24799]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:10.937 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:10 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:10.937 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:10 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:10.937 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:10 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:10.937 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:10 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:10.937 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:10 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:10.937 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:10 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:11:10.937 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:10 smithi062 ceph-mon[24799]: Reconfiguring mon.a (unknown last config time)... 2024-03-24T11:11:10.937 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:10 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T11:11:10.937 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:10 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T11:11:10.938 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:10 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:10.938 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:10 smithi062 ceph-mon[24799]: Reconfiguring daemon mon.a on smithi062 2024-03-24T11:11:10.938 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:10 smithi062 ceph-mon[24799]: from='client.? 172.21.15.139:0/468480649' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T11:11:10.938 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:10 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T11:11:12.226 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:11:12 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.62 - - [24/Mar/2024:11:11:12] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:11:12.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:11:12 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.139 - - [24/Mar/2024:11:11:12] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:11:13.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:13 smithi062 ceph-mon[24799]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:13.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:13 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:13.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:13 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:13.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:13 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:13.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:13 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:13.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:13 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:13.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:13 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T11:11:13.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:13 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T11:11:13.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:13 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:13.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:13 smithi062 ceph-mon[29666]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:13.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:13 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:13.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:13 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:13.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:13 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:13.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:13 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:13.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:13 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:13.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:13 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T11:11:13.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:13 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T11:11:13.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:13 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:13.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:13 smithi139 ceph-mon[25719]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:13.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:13 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:13.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:13 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:13.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:13 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:13.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:13 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:13.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:13 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:13.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:13 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T11:11:13.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:13 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T11:11:13.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:13 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:13.671 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:11:13 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.139 - - [24/Mar/2024:11:11:13] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:11:14.278 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:14 smithi062 ceph-mon[24799]: Reconfiguring mon.c (monmap changed)... 2024-03-24T11:11:14.278 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:14 smithi062 ceph-mon[24799]: Reconfiguring daemon mon.c on smithi062 2024-03-24T11:11:14.278 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:14 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:14.278 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:14 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:14.279 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:14 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:14.279 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:14 smithi062 ceph-mon[29666]: Reconfiguring mon.c (monmap changed)... 2024-03-24T11:11:14.279 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:14 smithi062 ceph-mon[29666]: Reconfiguring daemon mon.c on smithi062 2024-03-24T11:11:14.279 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:14 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:14.279 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:14 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:14.279 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:14 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:14.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:14 smithi139 ceph-mon[25719]: Reconfiguring mon.c (monmap changed)... 2024-03-24T11:11:14.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:14 smithi139 ceph-mon[25719]: Reconfiguring daemon mon.c on smithi062 2024-03-24T11:11:14.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:14 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:14.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:14 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:14.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:14 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:15.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:15 smithi139 ceph-mon[25719]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:15.121 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:15 smithi062 ceph-mon[29666]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:15.122 INFO:teuthology.orchestra.run.smithi062.stdout:# minimal ceph.conf for ebb89568-e9ce-11ee-95cd-87774f69a715 2024-03-24T11:11:15.122 INFO:teuthology.orchestra.run.smithi062.stdout:[global] 2024-03-24T11:11:15.122 INFO:teuthology.orchestra.run.smithi062.stdout: fsid = ebb89568-e9ce-11ee-95cd-87774f69a715 2024-03-24T11:11:15.122 INFO:teuthology.orchestra.run.smithi062.stdout: mon_host = [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] 2024-03-24T11:11:15.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:15 smithi062 ceph-mon[24799]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:15.422 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:11:15 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.62 - - [24/Mar/2024:11:11:15] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:11:15.730 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-24T11:11:15.731 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-24T11:11:15.731 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/ceph/ceph.conf 2024-03-24T11:11:15.801 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-24T11:11:15.801 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-24T11:11:15.868 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-03-24T11:11:15.869 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/ceph/ceph.conf 2024-03-24T11:11:15.906 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-03-24T11:11:15.906 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-24T11:11:15.976 INFO:tasks.cephadm:Adding mgr.y on smithi062 2024-03-24T11:11:15.977 INFO:tasks.cephadm:Adding mgr.x on smithi139 2024-03-24T11:11:15.977 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph orch apply mgr '2;smithi062=y;smithi139=x' 2024-03-24T11:11:16.320 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:16 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/3031292485' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:16.321 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:16 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:16.321 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:16 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:16.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:16 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/3031292485' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:16.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:16 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:16.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:16 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:16.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:16 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/3031292485' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:16.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:16 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:16.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:16 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:17.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:17 smithi062 ceph-mon[29666]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:17.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:17 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:17.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:17 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T11:11:17.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:17 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T11:11:17.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:17 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:17.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:17 smithi062 ceph-mon[24799]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:17.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:17 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:17.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:17 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T11:11:17.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:17 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T11:11:17.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:17 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:17.480 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:17 smithi139 ceph-mon[25719]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:17.480 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:17 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:17.480 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:17 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T11:11:17.480 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:17 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T11:11:17.480 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:17 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:18.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:18 smithi062 ceph-mon[24799]: Reconfiguring mon.b (monmap changed)... 2024-03-24T11:11:18.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:18 smithi062 ceph-mon[24799]: Reconfiguring daemon mon.b on smithi139 2024-03-24T11:11:18.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:18 smithi062 ceph-mon[29666]: Reconfiguring mon.b (monmap changed)... 2024-03-24T11:11:18.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:18 smithi062 ceph-mon[29666]: Reconfiguring daemon mon.b on smithi139 2024-03-24T11:11:18.530 INFO:teuthology.orchestra.run.smithi139.stdout:Scheduled mgr update... 2024-03-24T11:11:18.530 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:18 smithi139 ceph-mon[25719]: Reconfiguring mon.b (monmap changed)... 2024-03-24T11:11:18.530 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:18 smithi139 ceph-mon[25719]: Reconfiguring daemon mon.b on smithi139 2024-03-24T11:11:19.220 DEBUG:teuthology.orchestra.run.smithi139:mgr.x> sudo journalctl -f -n 0 -u ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mgr.x.service 2024-03-24T11:11:19.222 INFO:tasks.cephadm:Deploying OSDs... 2024-03-24T11:11:19.223 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-24T11:11:19.223 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/scratch_devs of=/dev/stdout 2024-03-24T11:11:19.241 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-24T11:11:19.242 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_1 2024-03-24T11:11:19.299 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-24T11:11:19.299 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-24T11:11:19.299 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-03-24T11:11:19.300 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-24T11:11:19.300 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-03-24T11:11:19.300 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-03-24 11:11:14.148337297 +0000 2024-03-24T11:11:19.300 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-03-24 11:08:38.681024018 +0000 2024-03-24T11:11:19.300 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-03-24 11:08:38.681024018 +0000 2024-03-24T11:11:19.300 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-03-24 11:08:38.681024018 +0000 2024-03-24T11:11:19.300 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-24T11:11:19.370 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-03-24T11:11:19.370 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-03-24T11:11:19.370 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000221712 s, 2.3 MB/s 2024-03-24T11:11:19.371 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-24T11:11:19.429 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_2 2024-03-24T11:11:19.473 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:19 smithi139 ceph-mon[25719]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:19.473 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:19 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:19.473 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:19 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:19.473 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:19 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:11:19.473 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:19 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:19.473 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:19 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:11:19.473 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:19 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:19.473 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:19 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T11:11:19.474 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:19 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-24T11:11:19.474 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:19 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T11:11:19.474 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:19 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:19.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:19 smithi062 ceph-mon[24799]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:19.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:19 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:19.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:19 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:19.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:19 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:11:19.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:19 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:19.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:19 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:11:19.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:19 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:19.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:19 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T11:11:19.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:19 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-24T11:11:19.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:19 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T11:11:19.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:19 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:19.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:19 smithi062 ceph-mon[29666]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:19.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:19 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:19.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:19 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:19.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:19 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:11:19.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:19 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:19.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:19 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:11:19.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:19 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:19.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:19 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T11:11:19.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:19 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-24T11:11:19.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:19 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T11:11:19.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:19 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:19.485 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-24T11:11:19.485 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-24T11:11:19.485 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-03-24T11:11:19.486 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-24T11:11:19.486 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-03-24T11:11:19.486 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-03-24 11:11:14.148337297 +0000 2024-03-24T11:11:19.486 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-03-24 11:08:39.031018387 +0000 2024-03-24T11:11:19.486 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-03-24 11:08:39.031018387 +0000 2024-03-24T11:11:19.486 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-03-24 11:08:39.031018387 +0000 2024-03-24T11:11:19.486 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-24T11:11:19.558 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-03-24T11:11:19.558 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-03-24T11:11:19.558 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000223573 s, 2.3 MB/s 2024-03-24T11:11:19.559 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-24T11:11:19.616 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_3 2024-03-24T11:11:19.674 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-24T11:11:19.674 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-24T11:11:19.674 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-03-24T11:11:19.675 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-24T11:11:19.675 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-03-24T11:11:19.675 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-03-24 11:11:14.148337297 +0000 2024-03-24T11:11:19.675 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-03-24 11:08:39.382012740 +0000 2024-03-24T11:11:19.675 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-03-24 11:08:39.382012740 +0000 2024-03-24T11:11:19.675 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-03-24 11:08:39.382012740 +0000 2024-03-24T11:11:19.675 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-24T11:11:19.742 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-03-24T11:11:19.742 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-03-24T11:11:19.742 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000234387 s, 2.2 MB/s 2024-03-24T11:11:19.744 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-24T11:11:19.801 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_4 2024-03-24T11:11:19.858 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-24T11:11:19.858 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-24T11:11:19.859 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-03-24T11:11:19.859 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-24T11:11:19.859 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-03-24T11:11:19.859 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-03-24 11:11:14.148337297 +0000 2024-03-24T11:11:19.859 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-03-24 11:08:39.731007124 +0000 2024-03-24T11:11:19.859 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-03-24 11:08:39.731007124 +0000 2024-03-24T11:11:19.859 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-03-24 11:08:39.731007124 +0000 2024-03-24T11:11:19.859 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-24T11:11:19.927 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-03-24T11:11:19.927 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-03-24T11:11:19.927 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000210364 s, 2.4 MB/s 2024-03-24T11:11:19.928 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-24T11:11:19.987 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-03-24T11:11:19.987 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/scratch_devs of=/dev/stdout 2024-03-24T11:11:20.004 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-24T11:11:20.004 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/vg_nvme/lv_1 2024-03-24T11:11:20.059 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-24T11:11:20.059 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-24T11:11:20.059 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-03-24T11:11:20.059 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-24T11:11:20.059 INFO:teuthology.orchestra.run.smithi139.stdout:Context: system_u:object_r:device_t:s0 2024-03-24T11:11:20.059 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2024-03-24 11:11:12.756858084 +0000 2024-03-24T11:11:20.059 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2024-03-24 11:08:38.665954984 +0000 2024-03-24T11:11:20.059 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2024-03-24 11:08:38.665954984 +0000 2024-03-24T11:11:20.059 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: 2024-03-24 11:08:38.665954984 +0000 2024-03-24T11:11:20.060 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-24T11:11:20.125 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2024-03-24T11:11:20.126 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2024-03-24T11:11:20.126 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.00019778 s, 2.6 MB/s 2024-03-24T11:11:20.127 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-24T11:11:20.184 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/vg_nvme/lv_2 2024-03-24T11:11:20.239 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-24T11:11:20.239 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-24T11:11:20.239 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-03-24T11:11:20.239 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-24T11:11:20.239 INFO:teuthology.orchestra.run.smithi139.stdout:Context: system_u:object_r:device_t:s0 2024-03-24T11:11:20.239 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2024-03-24 11:11:12.756858084 +0000 2024-03-24T11:11:20.240 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2024-03-24 11:08:39.017947678 +0000 2024-03-24T11:11:20.240 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2024-03-24 11:08:39.017947678 +0000 2024-03-24T11:11:20.240 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: 2024-03-24 11:08:39.017947678 +0000 2024-03-24T11:11:20.240 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-24T11:11:20.304 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2024-03-24T11:11:20.304 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2024-03-24T11:11:20.304 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.000189528 s, 2.7 MB/s 2024-03-24T11:11:20.306 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-24T11:11:20.361 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/vg_nvme/lv_3 2024-03-24T11:11:20.416 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-24T11:11:20.416 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-24T11:11:20.416 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-03-24T11:11:20.416 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-24T11:11:20.417 INFO:teuthology.orchestra.run.smithi139.stdout:Context: system_u:object_r:device_t:s0 2024-03-24T11:11:20.417 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2024-03-24 11:11:12.756858084 +0000 2024-03-24T11:11:20.417 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2024-03-24 11:08:39.351940745 +0000 2024-03-24T11:11:20.417 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2024-03-24 11:08:39.351940745 +0000 2024-03-24T11:11:20.417 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: 2024-03-24 11:08:39.351940745 +0000 2024-03-24T11:11:20.417 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-24T11:11:20.436 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:20 smithi139 ceph-mon[25719]: from='client.24107 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi062=y;smithi139=x", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:11:20.436 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:20 smithi139 ceph-mon[25719]: Saving service mgr spec with placement smithi062=y;smithi139=x;count:2 2024-03-24T11:11:20.436 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:20 smithi139 ceph-mon[25719]: Deploying daemon mgr.x on smithi139 2024-03-24T11:11:20.476 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:20 smithi062 ceph-mon[29666]: from='client.24107 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi062=y;smithi139=x", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:11:20.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:20 smithi062 ceph-mon[29666]: Saving service mgr spec with placement smithi062=y;smithi139=x;count:2 2024-03-24T11:11:20.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:20 smithi062 ceph-mon[29666]: Deploying daemon mgr.x on smithi139 2024-03-24T11:11:20.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:20 smithi062 ceph-mon[24799]: from='client.24107 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi062=y;smithi139=x", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:11:20.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:20 smithi062 ceph-mon[24799]: Saving service mgr spec with placement smithi062=y;smithi139=x;count:2 2024-03-24T11:11:20.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:20 smithi062 ceph-mon[24799]: Deploying daemon mgr.x on smithi139 2024-03-24T11:11:20.604 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2024-03-24T11:11:20.605 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2024-03-24T11:11:20.605 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.000224272 s, 2.3 MB/s 2024-03-24T11:11:20.606 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-24T11:11:20.621 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/vg_nvme/lv_4 2024-03-24T11:11:20.676 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-24T11:11:20.676 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-24T11:11:20.676 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-03-24T11:11:20.676 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-24T11:11:20.677 INFO:teuthology.orchestra.run.smithi139.stdout:Context: system_u:object_r:device_t:s0 2024-03-24T11:11:20.677 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2024-03-24 11:11:20.562693260 +0000 2024-03-24T11:11:20.677 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2024-03-24 11:08:39.695933605 +0000 2024-03-24T11:11:20.677 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2024-03-24 11:08:39.695933605 +0000 2024-03-24T11:11:20.677 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: 2024-03-24 11:08:39.695933605 +0000 2024-03-24T11:11:20.677 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-24T11:11:20.770 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2024-03-24T11:11:20.771 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2024-03-24T11:11:20.771 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.000229303 s, 2.2 MB/s 2024-03-24T11:11:20.772 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-24T11:11:20.787 INFO:tasks.cephadm:Deploying osd.0 on smithi062 with /dev/vg_nvme/lv_4... 2024-03-24T11:11:20.787 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-24T11:11:20.980 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:11:20 smithi139 systemd[1]: Starting Ceph mgr.x for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:11:21.238 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:21 smithi139 ceph-mon[25719]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:21.239 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:11:20 smithi139 podman[27216]: 2024-03-24 11:11:20.979602397 +0000 UTC m=+0.089797768 container create 9bbcf96d02d60a5640bccf72a7c875039e7cecc9b10a40d9e823bfb4af0679c2 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x, io.openshift.expose-services=, version=8, RELEASE=HEAD, CEPH_POINT_RELEASE=-17.2.0, io.buildah.version=1.19.8, maintainer=Guillaume Abrioux , build-date=2022-05-03T08:36:31.336870, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, GIT_REPO=https://github.com/ceph/ceph-container.git, com.redhat.component=centos-stream-container, distribution-scope=public, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., architecture=x86_64, release=754, GIT_CLEAN=True, vendor=Red Hat, Inc., com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, ceph=True, GIT_BRANCH=HEAD, io.k8s.display-name=CentOS Stream 8, vcs-type=git, name=centos-stream, io.openshift.tags=base centos centos-stream, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac) 2024-03-24T11:11:21.239 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:11:21 smithi139 podman[27216]: 2024-03-24 11:11:20.907807191 +0000 UTC m=+0.018002575 image pull e1d6a67b021eb077ee22bf650f1a9fb1980a2cf5c36bdb9cba9eac6de8f702d9 quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a 2024-03-24T11:11:21.239 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:11:21 smithi139 podman[27216]: 2024-03-24 11:11:21.21544227 +0000 UTC m=+0.325637651 container init 9bbcf96d02d60a5640bccf72a7c875039e7cecc9b10a40d9e823bfb4af0679c2 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x, version=8, CEPH_POINT_RELEASE=-17.2.0, io.openshift.tags=base centos centos-stream, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, GIT_CLEAN=True, name=centos-stream, distribution-scope=public, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.k8s.display-name=CentOS Stream 8, build-date=2022-05-03T08:36:31.336870, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, architecture=x86_64, io.buildah.version=1.19.8, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, vendor=Red Hat, Inc., com.redhat.component=centos-stream-container, GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, io.openshift.expose-services=, vcs-type=git, release=754, RELEASE=HEAD, maintainer=Guillaume Abrioux ) 2024-03-24T11:11:21.239 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:11:21 smithi139 podman[27216]: 2024-03-24 11:11:21.218312028 +0000 UTC m=+0.328507408 container start 9bbcf96d02d60a5640bccf72a7c875039e7cecc9b10a40d9e823bfb4af0679c2 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, RELEASE=HEAD, maintainer=Guillaume Abrioux , io.k8s.display-name=CentOS Stream 8, GIT_REPO=https://github.com/ceph/ceph-container.git, CEPH_POINT_RELEASE=-17.2.0, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.openshift.expose-services=, name=centos-stream, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, vendor=Red Hat, Inc., io.openshift.tags=base centos centos-stream, io.buildah.version=1.19.8, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, com.redhat.component=centos-stream-container, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, GIT_BRANCH=HEAD, version=8, ceph=True, architecture=x86_64, release=754, GIT_CLEAN=True, vcs-type=git, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, distribution-scope=public, build-date=2022-05-03T08:36:31.336870) 2024-03-24T11:11:21.239 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:11:21 smithi139 ceph-mgr[27235]: -- 172.21.15.139:0/1532740957 <== mon.2 v2:172.21.15.139:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56115fb26340 con 0x56115fafe800 2024-03-24T11:11:21.360 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:21 smithi062 ceph-mon[24799]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:21.361 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:21 smithi062 ceph-mon[29666]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:21.607 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:11:21 smithi139 bash[27216]: 9bbcf96d02d60a5640bccf72a7c875039e7cecc9b10a40d9e823bfb4af0679c2 2024-03-24T11:11:21.608 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:11:21 smithi139 systemd[1]: Started Ceph mgr.x for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:11:22.107 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:11:21 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: 2024-03-24T11:11:21.696+0000 7fb39949c000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-24T11:11:22.108 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:11:21 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: 2024-03-24T11:11:21.785+0000 7fb39949c000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-24T11:11:22.695 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:22 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:22.695 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:22 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:11:22.695 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:22 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:22.695 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:22 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:11:22.695 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:22 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:22.695 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:22 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:11:22.695 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:22 smithi062 ceph-mon[24799]: Reconfiguring mgr.y (unknown last config time)... 2024-03-24T11:11:22.695 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:22 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T11:11:22.695 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:22 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T11:11:22.695 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:22 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:22.695 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:22 smithi062 ceph-mon[24799]: Reconfiguring daemon mgr.y on smithi062 2024-03-24T11:11:22.695 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:22 smithi062 ceph-mon[24799]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:22.696 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:22 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:22.696 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:22 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:11:22.696 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:22 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:22.696 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:22 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:11:22.696 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:22 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:22.696 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:22 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:11:22.696 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:22 smithi062 ceph-mon[29666]: Reconfiguring mgr.y (unknown last config time)... 2024-03-24T11:11:22.696 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:22 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T11:11:22.697 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:22 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T11:11:22.697 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:22 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:22.697 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:22 smithi062 ceph-mon[29666]: Reconfiguring daemon mgr.y on smithi062 2024-03-24T11:11:22.697 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:22 smithi062 ceph-mon[29666]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:22.710 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:11:22 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: 2024-03-24T11:11:22.377+0000 7fb39949c000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-24T11:11:22.711 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:11:22 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: 2024-03-24T11:11:22.477+0000 7fb39949c000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-24T11:11:22.711 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:11:22 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: 2024-03-24T11:11:22.561+0000 7fb39949c000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-24T11:11:22.711 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:22 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:22.711 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:22 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:11:22.712 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:22 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:22.712 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:22 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:11:22.712 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:22 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:22.712 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:22 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:11:22.712 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:22 smithi139 ceph-mon[25719]: Reconfiguring mgr.y (unknown last config time)... 2024-03-24T11:11:22.713 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:22 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T11:11:22.713 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:22 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T11:11:22.713 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:22 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:22.713 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:22 smithi139 ceph-mon[25719]: Reconfiguring daemon mgr.y on smithi062 2024-03-24T11:11:22.713 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:22 smithi139 ceph-mon[25719]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:22.986 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:11:22 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: 2024-03-24T11:11:22.715+0000 7fb39949c000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-24T11:11:22.986 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:11:22 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: 2024-03-24T11:11:22.797+0000 7fb39949c000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-24T11:11:22.986 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:11:22 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: 2024-03-24T11:11:22.904+0000 7fb39949c000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-24T11:11:22.987 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:11:22 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: 2024-03-24T11:11:22.985+0000 7fb39949c000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-24T11:11:23.242 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:11:23 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: 2024-03-24T11:11:23.158+0000 7fb39949c000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-24T11:11:23.243 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:11:23 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: 2024-03-24T11:11:23.242+0000 7fb39949c000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-24T11:11:23.665 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-24T11:11:23.665 INFO:teuthology.orchestra.run.smithi062.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-24T11:11:23.665 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10+0 records in 2024-03-24T11:11:23.666 INFO:teuthology.orchestra.run.smithi062.stderr:10+0 records out 2024-03-24T11:11:23.666 INFO:teuthology.orchestra.run.smithi062.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.017206 s, 609 MB/s 2024-03-24T11:11:23.666 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 2024-03-24T11:11:23.666 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping successful for: 2024-03-24T11:11:24.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:24 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:24.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:24 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:11:24.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:24 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:24.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:24 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:11:24.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:24 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:24.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:24 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:11:24.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:24 smithi062 ceph-mon[24799]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:24.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:24 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-24T11:11:24.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:24 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-24T11:11:24.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:24 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:24.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:24 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:24.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:24 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:11:24.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:24 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:24.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:24 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:11:24.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:24 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:24.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:24 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:11:24.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:24 smithi062 ceph-mon[29666]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:24.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:24 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-24T11:11:24.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:24 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-24T11:11:24.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:24 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:24.319 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi062:vg_nvme/lv_4 2024-03-24T11:11:24.466 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:11:24 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: 2024-03-24T11:11:24.328+0000 7fb39949c000 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-24T11:11:24.467 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:11:24 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: 2024-03-24T11:11:24.465+0000 7fb39949c000 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-24T11:11:24.467 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:24 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:24.467 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:24 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:11:24.467 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:24 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:24.467 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:24 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:11:24.467 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:24 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:24.467 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:24 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:11:24.467 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:24 smithi139 ceph-mon[25719]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:24.467 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:24 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-24T11:11:24.467 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:24 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-24T11:11:24.467 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:24 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:24.748 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:11:24 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: 2024-03-24T11:11:24.662+0000 7fb39949c000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-24T11:11:24.749 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:11:24 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: 2024-03-24T11:11:24.748+0000 7fb39949c000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-24T11:11:25.108 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:11:24 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: 2024-03-24T11:11:24.883+0000 7fb39949c000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-24T11:11:25.418 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:11:25 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.139 - - [24/Mar/2024:11:11:25] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:11:25.422 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:11:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: 2024-03-24T11:11:25.214+0000 7fb39949c000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-24T11:11:25.712 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:11:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: 2024-03-24T11:11:25.438+0000 7fb39949c000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-24T11:11:25.713 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:11:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: 2024-03-24T11:11:25.544+0000 7fb39949c000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-24T11:11:25.713 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:11:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: 2024-03-24T11:11:25.712+0000 7fb39949c000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-24T11:11:25.722 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:11:25 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.139 - - [24/Mar/2024:11:11:25] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:11:26.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:26 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:26.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:26 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:26.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:26 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:11:26.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:26 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:26.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:26 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:11:26.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:26 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:26.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:26 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:26.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:26 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:26.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:26 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T11:11:26.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:26 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T11:11:26.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:26 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:26.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:26 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:26.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:26 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:26.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:26 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:11:26.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:26 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:26.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:26 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:11:26.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:26 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:26.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:26 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:26.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:26 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:26.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:26 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T11:11:26.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:26 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T11:11:26.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:26 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:26.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:26 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:26.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:26 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:26.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:26 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:11:26.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:26 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:26.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:26 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:11:26.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:26 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:26.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:26 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:26.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:26 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:26.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:26 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T11:11:26.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:26 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T11:11:26.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:26 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:26.913 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:11:26 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: 2024-03-24T11:11:26.621+0000 7fb39949c000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-24T11:11:26.913 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:11:26 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: 2024-03-24T11:11:26.912+0000 7fb39949c000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-24T11:11:27.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:27 smithi139 ceph-mon[25719]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:27.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:27 smithi139 ceph-mon[25719]: from='client.24118 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:11:27.358 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:11:27 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: 2024-03-24T11:11:27.006+0000 7fb39949c000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-24T11:11:27.358 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:11:27 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: 2024-03-24T11:11:27.153+0000 7fb39949c000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-24T11:11:27.404 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:27 smithi062 ceph-mon[24799]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:27.405 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:27 smithi062 ceph-mon[24799]: from='client.24118 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:11:27.405 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:27 smithi062 ceph-mon[29666]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:27.405 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:27 smithi062 ceph-mon[29666]: from='client.24118 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:11:28.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:28 smithi062 ceph-mon[24799]: Standby manager daemon x started 2024-03-24T11:11:28.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:28 smithi062 ceph-mon[24799]: from='mgr.? 172.21.15.139:0/3935563113' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-24T11:11:28.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:28 smithi062 ceph-mon[24799]: from='mgr.? 172.21.15.139:0/3935563113' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T11:11:28.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:28 smithi062 ceph-mon[24799]: from='mgr.? 172.21.15.139:0/3935563113' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-24T11:11:28.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:28 smithi062 ceph-mon[24799]: from='mgr.? 172.21.15.139:0/3935563113' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T11:11:28.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:28 smithi062 ceph-mon[29666]: Standby manager daemon x started 2024-03-24T11:11:28.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:28 smithi062 ceph-mon[29666]: from='mgr.? 172.21.15.139:0/3935563113' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-24T11:11:28.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:28 smithi062 ceph-mon[29666]: from='mgr.? 172.21.15.139:0/3935563113' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T11:11:28.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:28 smithi062 ceph-mon[29666]: from='mgr.? 172.21.15.139:0/3935563113' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-24T11:11:28.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:28 smithi062 ceph-mon[29666]: from='mgr.? 172.21.15.139:0/3935563113' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T11:11:28.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:28 smithi139 ceph-mon[25719]: Standby manager daemon x started 2024-03-24T11:11:28.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:28 smithi139 ceph-mon[25719]: from='mgr.? 172.21.15.139:0/3935563113' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-24T11:11:28.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:28 smithi139 ceph-mon[25719]: from='mgr.? 172.21.15.139:0/3935563113' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T11:11:28.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:28 smithi139 ceph-mon[25719]: from='mgr.? 172.21.15.139:0/3935563113' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-24T11:11:28.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:28 smithi139 ceph-mon[25719]: from='mgr.? 172.21.15.139:0/3935563113' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T11:11:29.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:29 smithi062 ceph-mon[24799]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:29.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:29 smithi062 ceph-mon[24799]: mgrmap e14: y(active, since 65s), standbys: x 2024-03-24T11:11:29.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:29 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-24T11:11:29.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:29 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/1955800515' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83103b18-933d-437b-9762-583f85bc1776"}]: dispatch 2024-03-24T11:11:29.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:29 smithi062 ceph-mon[24799]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83103b18-933d-437b-9762-583f85bc1776"}]: dispatch 2024-03-24T11:11:29.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:29 smithi062 ceph-mon[24799]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83103b18-933d-437b-9762-583f85bc1776"}]': finished 2024-03-24T11:11:29.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:29 smithi062 ceph-mon[24799]: osdmap e5: 1 total, 0 up, 1 in 2024-03-24T11:11:29.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:29 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T11:11:29.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:29 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/2795779806' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T11:11:29.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:29 smithi062 ceph-mon[29666]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:29.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:29 smithi062 ceph-mon[29666]: mgrmap e14: y(active, since 65s), standbys: x 2024-03-24T11:11:29.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:29 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-24T11:11:29.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:29 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/1955800515' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83103b18-933d-437b-9762-583f85bc1776"}]: dispatch 2024-03-24T11:11:29.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:29 smithi062 ceph-mon[29666]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83103b18-933d-437b-9762-583f85bc1776"}]: dispatch 2024-03-24T11:11:29.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:29 smithi062 ceph-mon[29666]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83103b18-933d-437b-9762-583f85bc1776"}]': finished 2024-03-24T11:11:29.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:29 smithi062 ceph-mon[29666]: osdmap e5: 1 total, 0 up, 1 in 2024-03-24T11:11:29.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:29 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T11:11:29.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:29 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/2795779806' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T11:11:29.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:29 smithi139 ceph-mon[25719]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:29.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:29 smithi139 ceph-mon[25719]: mgrmap e14: y(active, since 65s), standbys: x 2024-03-24T11:11:29.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:29 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-24T11:11:29.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:29 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/1955800515' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83103b18-933d-437b-9762-583f85bc1776"}]: dispatch 2024-03-24T11:11:29.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:29 smithi139 ceph-mon[25719]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83103b18-933d-437b-9762-583f85bc1776"}]: dispatch 2024-03-24T11:11:29.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:29 smithi139 ceph-mon[25719]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83103b18-933d-437b-9762-583f85bc1776"}]': finished 2024-03-24T11:11:29.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:29 smithi139 ceph-mon[25719]: osdmap e5: 1 total, 0 up, 1 in 2024-03-24T11:11:29.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:29 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T11:11:29.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:29 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/2795779806' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T11:11:31.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:31 smithi062 ceph-mon[24799]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:31.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:31 smithi062 ceph-mon[29666]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:31.607 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:31 smithi139 ceph-mon[25719]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:33.427 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:33 smithi139 ceph-mon[25719]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:33.435 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:33 smithi062 ceph-mon[24799]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:33.435 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:33 smithi062 ceph-mon[29666]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:34.477 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:11:34 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.62 - - [24/Mar/2024:11:11:34] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:11:35.457 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:35 smithi062 ceph-mon[24799]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:35.457 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:35 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:35.457 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:35 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:35.458 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:35 smithi062 ceph-mon[29666]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:35.458 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:35 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:35.458 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:35 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:35.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:35 smithi139 ceph-mon[25719]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:35.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:35 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:35.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:35 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:36.427 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:36 smithi062 ceph-mon[24799]: Detected new or changed devices on smithi062 2024-03-24T11:11:36.428 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:36 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-24T11:11:36.428 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:36 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:36.428 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:36 smithi062 ceph-mon[29666]: Detected new or changed devices on smithi062 2024-03-24T11:11:36.428 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:36 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-24T11:11:36.428 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:36 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:36.607 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:36 smithi139 ceph-mon[25719]: Detected new or changed devices on smithi062 2024-03-24T11:11:36.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:36 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-24T11:11:36.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:36 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:37.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:37 smithi062 ceph-mon[24799]: Deploying daemon osd.0 on smithi062 2024-03-24T11:11:37.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:37 smithi062 ceph-mon[24799]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:37.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:37 smithi062 ceph-mon[29666]: Deploying daemon osd.0 on smithi062 2024-03-24T11:11:37.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:37 smithi062 ceph-mon[29666]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:37.607 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:37 smithi139 ceph-mon[25719]: Deploying daemon osd.0 on smithi062 2024-03-24T11:11:37.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:37 smithi139 ceph-mon[25719]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:38.911 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:38 smithi062 ceph-mon[24799]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:38.911 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:38 smithi062 ceph-mon[29666]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:39.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:38 smithi139 ceph-mon[25719]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:40.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:40 smithi062 ceph-mon[24799]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:40.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:40 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:40.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:40 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:11:40.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:40 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:40.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:40 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:11:40.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:40 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:40.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:40 smithi062 ceph-mon[29666]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:40.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:40 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:40.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:40 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:11:40.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:40 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:40.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:40 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:11:40.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:40 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:41.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:40 smithi139 ceph-mon[25719]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:41.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:40 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:41.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:40 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:11:41.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:40 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:41.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:40 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:11:41.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:40 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:43.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:42 smithi139 ceph-mon[25719]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:43.162 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:42 smithi062 ceph-mon[24799]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:43.162 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:42 smithi062 ceph-mon[29666]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:43.191 INFO:teuthology.orchestra.run.smithi062.stdout:Created osd(s) 0 on host 'smithi062' 2024-03-24T11:11:43.476 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:11:43 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.139 - - [24/Mar/2024:11:11:43] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:11:43.856 DEBUG:teuthology.orchestra.run.smithi062:osd.0> sudo journalctl -f -n 0 -u ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.0.service 2024-03-24T11:11:43.859 INFO:tasks.cephadm:Deploying osd.1 on smithi062 with /dev/vg_nvme/lv_3... 2024-03-24T11:11:43.860 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-24T11:11:44.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:44 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:44.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:44 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:11:44.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:44 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:44.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:44 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:11:44.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:44 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:11:44.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:44 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:44.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:44 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:44.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:44 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:44.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:44 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:44.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:44 smithi062 ceph-mon[24799]: from='osd.0 [v2:172.21.15.62:6802/1287363541,v1:172.21.15.62:6803/1287363541]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-24T11:11:44.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:44 smithi062 ceph-mon[24799]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-24T11:11:44.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:44 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:44.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:44 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:11:44.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:44 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:44.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:44 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:11:44.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:44 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:11:44.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:44 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:44.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:44 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:44.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:44 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:44.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:44 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:44.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:44 smithi062 ceph-mon[29666]: from='osd.0 [v2:172.21.15.62:6802/1287363541,v1:172.21.15.62:6803/1287363541]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-24T11:11:44.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:44 smithi062 ceph-mon[29666]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-24T11:11:44.519 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:44 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:44.519 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:44 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:11:44.519 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:44 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:44.519 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:44 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:11:44.519 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:44 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:11:44.519 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:44 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:44.519 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:44 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:44.519 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:44 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:44.519 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:44 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:44.520 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:44 smithi139 ceph-mon[25719]: from='osd.0 [v2:172.21.15.62:6802/1287363541,v1:172.21.15.62:6803/1287363541]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-24T11:11:44.520 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:44 smithi139 ceph-mon[25719]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-24T11:11:45.451 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:45 smithi062 ceph-mon[24799]: Adjusting osd_memory_target on smithi062 to 15040M 2024-03-24T11:11:45.451 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:45 smithi062 ceph-mon[24799]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:45.451 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:45 smithi062 ceph-mon[24799]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-24T11:11:45.451 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:45 smithi062 ceph-mon[24799]: osdmap e6: 1 total, 0 up, 1 in 2024-03-24T11:11:45.452 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:45 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T11:11:45.452 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:45 smithi062 ceph-mon[24799]: from='osd.0 [v2:172.21.15.62:6802/1287363541,v1:172.21.15.62:6803/1287363541]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:11:45.452 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:45 smithi062 ceph-mon[24799]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:11:45.452 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:45 smithi062 ceph-mon[29666]: Adjusting osd_memory_target on smithi062 to 15040M 2024-03-24T11:11:45.452 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:45 smithi062 ceph-mon[29666]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:45.452 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:45 smithi062 ceph-mon[29666]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-24T11:11:45.452 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:45 smithi062 ceph-mon[29666]: osdmap e6: 1 total, 0 up, 1 in 2024-03-24T11:11:45.452 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:45 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T11:11:45.452 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:45 smithi062 ceph-mon[29666]: from='osd.0 [v2:172.21.15.62:6802/1287363541,v1:172.21.15.62:6803/1287363541]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:11:45.452 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:45 smithi062 ceph-mon[29666]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:11:45.453 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:11:45 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0[32974]: 2024-03-24T11:11:45.271+0000 7fa9d9bda700 -1 osd.0 0 waiting for initial osdmap 2024-03-24T11:11:45.453 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:11:45 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0[32974]: 2024-03-24T11:11:45.275+0000 7fa9d256d700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T11:11:45.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:45 smithi139 ceph-mon[25719]: Adjusting osd_memory_target on smithi062 to 15040M 2024-03-24T11:11:45.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:45 smithi139 ceph-mon[25719]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:45.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:45 smithi139 ceph-mon[25719]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-24T11:11:45.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:45 smithi139 ceph-mon[25719]: osdmap e6: 1 total, 0 up, 1 in 2024-03-24T11:11:45.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:45 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T11:11:45.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:45 smithi139 ceph-mon[25719]: from='osd.0 [v2:172.21.15.62:6802/1287363541,v1:172.21.15.62:6803/1287363541]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:11:45.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:45 smithi139 ceph-mon[25719]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:11:45.840 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-24T11:11:45.841 INFO:teuthology.orchestra.run.smithi062.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-24T11:11:45.841 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10+0 records in 2024-03-24T11:11:45.841 INFO:teuthology.orchestra.run.smithi062.stderr:10+0 records out 2024-03-24T11:11:45.841 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0154081 s, 681 MB/s 2024-03-24T11:11:45.841 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping successful for: 2024-03-24T11:11:46.388 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:46 smithi062 ceph-mon[24799]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-03-24T11:11:46.388 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:46 smithi062 ceph-mon[24799]: osdmap e7: 1 total, 0 up, 1 in 2024-03-24T11:11:46.388 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:46 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T11:11:46.388 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:46 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T11:11:46.388 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:46 smithi062 ceph-mon[24799]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:46.388 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:46 smithi062 ceph-mon[29666]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-03-24T11:11:46.388 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:46 smithi062 ceph-mon[29666]: osdmap e7: 1 total, 0 up, 1 in 2024-03-24T11:11:46.388 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:46 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T11:11:46.389 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:46 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T11:11:46.389 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:46 smithi062 ceph-mon[29666]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:46.411 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi062:vg_nvme/lv_3 2024-03-24T11:11:46.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:46 smithi139 ceph-mon[25719]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-03-24T11:11:46.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:46 smithi139 ceph-mon[25719]: osdmap e7: 1 total, 0 up, 1 in 2024-03-24T11:11:46.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:46 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T11:11:46.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:46 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T11:11:46.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:46 smithi139 ceph-mon[25719]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T11:11:47.561 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:47 smithi062 ceph-mon[24799]: purged_snaps scrub starts 2024-03-24T11:11:47.561 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:47 smithi062 ceph-mon[24799]: purged_snaps scrub ok 2024-03-24T11:11:47.561 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:47 smithi062 ceph-mon[24799]: osd.0 [v2:172.21.15.62:6802/1287363541,v1:172.21.15.62:6803/1287363541] boot 2024-03-24T11:11:47.561 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:47 smithi062 ceph-mon[24799]: osdmap e8: 1 total, 1 up, 1 in 2024-03-24T11:11:47.561 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:47 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T11:11:47.561 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:47 smithi062 ceph-mon[29666]: purged_snaps scrub starts 2024-03-24T11:11:47.561 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:47 smithi062 ceph-mon[29666]: purged_snaps scrub ok 2024-03-24T11:11:47.561 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:47 smithi062 ceph-mon[29666]: osd.0 [v2:172.21.15.62:6802/1287363541,v1:172.21.15.62:6803/1287363541] boot 2024-03-24T11:11:47.562 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:47 smithi062 ceph-mon[29666]: osdmap e8: 1 total, 1 up, 1 in 2024-03-24T11:11:47.562 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:47 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T11:11:47.607 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:47 smithi139 ceph-mon[25719]: purged_snaps scrub starts 2024-03-24T11:11:47.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:47 smithi139 ceph-mon[25719]: purged_snaps scrub ok 2024-03-24T11:11:47.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:47 smithi139 ceph-mon[25719]: osd.0 [v2:172.21.15.62:6802/1287363541,v1:172.21.15.62:6803/1287363541] boot 2024-03-24T11:11:47.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:47 smithi139 ceph-mon[25719]: osdmap e8: 1 total, 1 up, 1 in 2024-03-24T11:11:47.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:47 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T11:11:48.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:48 smithi062 ceph-mon[24799]: osdmap e9: 1 total, 1 up, 1 in 2024-03-24T11:11:48.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:48 smithi062 ceph-mon[24799]: pgmap v41: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-03-24T11:11:48.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:48 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T11:11:48.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:48 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T11:11:48.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:48 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:48.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:48 smithi062 ceph-mon[29666]: osdmap e9: 1 total, 1 up, 1 in 2024-03-24T11:11:48.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:48 smithi062 ceph-mon[29666]: pgmap v41: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-03-24T11:11:48.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:48 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T11:11:48.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:48 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T11:11:48.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:48 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:48.607 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:48 smithi139 ceph-mon[25719]: osdmap e9: 1 total, 1 up, 1 in 2024-03-24T11:11:48.607 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:48 smithi139 ceph-mon[25719]: pgmap v41: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-03-24T11:11:48.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:48 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T11:11:48.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:48 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T11:11:48.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:48 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:49.975 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:49 smithi062 ceph-mon[24799]: from='client.24145 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:11:49.975 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:49 smithi062 ceph-mon[29666]: from='client.24145 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:11:50.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:49 smithi139 ceph-mon[25719]: from='client.24145 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:11:51.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:50 smithi139 ceph-mon[25719]: pgmap v42: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-03-24T11:11:51.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:50 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:51.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:50 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:51.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:50 smithi062 ceph-mon[24799]: pgmap v42: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-03-24T11:11:51.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:50 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:51.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:50 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:51.228 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:11:50 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.62 - - [24/Mar/2024:11:11:50] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:11:51.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:50 smithi062 ceph-mon[29666]: pgmap v42: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-03-24T11:11:51.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:50 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:51.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:50 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:11:52.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:51 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/847133526' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "515c2c9b-7bc4-4b00-9be7-8773cf551306"}]: dispatch 2024-03-24T11:11:52.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:51 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/847133526' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "515c2c9b-7bc4-4b00-9be7-8773cf551306"}]': finished 2024-03-24T11:11:52.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:51 smithi139 ceph-mon[25719]: osdmap e10: 2 total, 1 up, 2 in 2024-03-24T11:11:52.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:51 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T11:11:52.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:51 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/1443469733' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T11:11:52.187 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:51 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/847133526' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "515c2c9b-7bc4-4b00-9be7-8773cf551306"}]: dispatch 2024-03-24T11:11:52.187 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:51 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/847133526' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "515c2c9b-7bc4-4b00-9be7-8773cf551306"}]': finished 2024-03-24T11:11:52.187 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:51 smithi062 ceph-mon[29666]: osdmap e10: 2 total, 1 up, 2 in 2024-03-24T11:11:52.187 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:51 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T11:11:52.187 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:51 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/1443469733' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T11:11:52.187 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:51 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/847133526' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "515c2c9b-7bc4-4b00-9be7-8773cf551306"}]: dispatch 2024-03-24T11:11:52.188 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:51 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/847133526' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "515c2c9b-7bc4-4b00-9be7-8773cf551306"}]': finished 2024-03-24T11:11:52.188 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:51 smithi062 ceph-mon[24799]: osdmap e10: 2 total, 1 up, 2 in 2024-03-24T11:11:52.188 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:51 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T11:11:52.188 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:51 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/1443469733' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T11:11:53.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:52 smithi139 ceph-mon[25719]: pgmap v44: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-03-24T11:11:53.476 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:52 smithi062 ceph-mon[29666]: pgmap v44: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-03-24T11:11:53.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:52 smithi062 ceph-mon[24799]: pgmap v44: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-03-24T11:11:55.214 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:54 smithi062 ceph-mon[24799]: pgmap v45: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-03-24T11:11:55.215 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:54 smithi062 ceph-mon[29666]: pgmap v45: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-03-24T11:11:55.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:54 smithi139 ceph-mon[25719]: pgmap v45: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-03-24T11:11:57.075 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:56 smithi062 ceph-mon[24799]: pgmap v46: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-03-24T11:11:57.075 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:56 smithi062 ceph-mon[29666]: pgmap v46: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-03-24T11:11:57.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:56 smithi139 ceph-mon[25719]: pgmap v46: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-03-24T11:11:58.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:58 smithi062 ceph-mon[24799]: pgmap v47: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-03-24T11:11:58.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:58 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-24T11:11:58.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:58 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:58.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:11:58 smithi062 ceph-mon[24799]: Deploying daemon osd.1 on smithi062 2024-03-24T11:11:58.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:58 smithi062 ceph-mon[29666]: pgmap v47: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-03-24T11:11:58.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:58 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-24T11:11:58.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:58 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:58.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:11:58 smithi062 ceph-mon[29666]: Deploying daemon osd.1 on smithi062 2024-03-24T11:11:59.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:58 smithi139 ceph-mon[25719]: pgmap v47: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-03-24T11:11:59.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:58 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-24T11:11:59.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:58 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:11:59.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:11:58 smithi139 ceph-mon[25719]: Deploying daemon osd.1 on smithi062 2024-03-24T11:12:00.960 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:00 smithi062 ceph-mon[24799]: pgmap v48: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-03-24T11:12:00.961 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:00 smithi062 ceph-mon[29666]: pgmap v48: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-03-24T11:12:01.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:00 smithi139 ceph-mon[25719]: pgmap v48: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-03-24T11:12:03.072 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:02 smithi062 ceph-mon[24799]: pgmap v49: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-03-24T11:12:03.072 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:02 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:03.072 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:02 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:12:03.073 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:02 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:12:03.073 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:02 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:12:03.073 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:02 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:03.073 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:02 smithi062 ceph-mon[29666]: pgmap v49: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-03-24T11:12:03.073 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:02 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:03.073 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:02 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:12:03.073 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:02 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:12:03.073 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:02 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:12:03.073 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:02 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:03.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:02 smithi139 ceph-mon[25719]: pgmap v49: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-03-24T11:12:03.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:02 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:03.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:02 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:12:03.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:02 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:12:03.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:02 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:12:03.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:02 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:03.726 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:12:03 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.139 - - [24/Mar/2024:11:12:03] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:12:04.374 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:04 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:04.374 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:04 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:04.374 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:04 smithi139 ceph-mon[25719]: pgmap v50: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-03-24T11:12:04.659 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:04 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:04.659 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:04 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:04.659 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:04 smithi062 ceph-mon[24799]: pgmap v50: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-03-24T11:12:04.659 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:04 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:04.659 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:04 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:04.660 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:04 smithi062 ceph-mon[29666]: pgmap v50: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-03-24T11:12:05.206 INFO:teuthology.orchestra.run.smithi062.stdout:Created osd(s) 1 on host 'smithi062' 2024-03-24T11:12:05.850 DEBUG:teuthology.orchestra.run.smithi062:osd.1> sudo journalctl -f -n 0 -u ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.1.service 2024-03-24T11:12:05.852 INFO:tasks.cephadm:Deploying osd.2 on smithi062 with /dev/vg_nvme/lv_2... 2024-03-24T11:12:05.852 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-24T11:12:06.283 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:06 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:06.284 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:06 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:12:06.284 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:06 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:12:06.284 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:06 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:12:06.284 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:06 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:12:06.284 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:06 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:12:06.284 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:06 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:06.284 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:06 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:06.284 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:06 smithi139 ceph-mon[25719]: from='osd.1 [v2:172.21.15.62:6810/1156951329,v1:172.21.15.62:6811/1156951329]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-24T11:12:06.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:06 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:06.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:06 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:12:06.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:06 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:12:06.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:06 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:12:06.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:06 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:12:06.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:06 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:12:06.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:06 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:06.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:06 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:06.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:06 smithi062 ceph-mon[24799]: from='osd.1 [v2:172.21.15.62:6810/1156951329,v1:172.21.15.62:6811/1156951329]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-24T11:12:06.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:06 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:06.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:06 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:12:06.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:06 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:12:06.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:06 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:12:06.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:06 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:12:06.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:06 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:12:06.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:06 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:06.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:06 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:06.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:06 smithi062 ceph-mon[29666]: from='osd.1 [v2:172.21.15.62:6810/1156951329,v1:172.21.15.62:6811/1156951329]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-24T11:12:07.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:07 smithi062 ceph-mon[24799]: Adjusting osd_memory_target on smithi062 to 7520M 2024-03-24T11:12:07.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:07 smithi062 ceph-mon[24799]: pgmap v51: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-03-24T11:12:07.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:07 smithi062 ceph-mon[24799]: from='osd.1 [v2:172.21.15.62:6810/1156951329,v1:172.21.15.62:6811/1156951329]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-24T11:12:07.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:07 smithi062 ceph-mon[24799]: osdmap e11: 2 total, 1 up, 2 in 2024-03-24T11:12:07.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:07 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T11:12:07.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:07 smithi062 ceph-mon[24799]: from='osd.1 [v2:172.21.15.62:6810/1156951329,v1:172.21.15.62:6811/1156951329]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:12:07.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:07 smithi062 ceph-mon[29666]: Adjusting osd_memory_target on smithi062 to 7520M 2024-03-24T11:12:07.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:07 smithi062 ceph-mon[29666]: pgmap v51: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-03-24T11:12:07.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:07 smithi062 ceph-mon[29666]: from='osd.1 [v2:172.21.15.62:6810/1156951329,v1:172.21.15.62:6811/1156951329]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-24T11:12:07.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:07 smithi062 ceph-mon[29666]: osdmap e11: 2 total, 1 up, 2 in 2024-03-24T11:12:07.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:07 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T11:12:07.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:07 smithi062 ceph-mon[29666]: from='osd.1 [v2:172.21.15.62:6810/1156951329,v1:172.21.15.62:6811/1156951329]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:12:07.478 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:12:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1[35856]: 2024-03-24T11:12:07.246+0000 7f56215d6700 -1 osd.1 0 waiting for initial osdmap 2024-03-24T11:12:07.478 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:12:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1[35856]: 2024-03-24T11:12:07.249+0000 7f561df71700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T11:12:07.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:07 smithi139 ceph-mon[25719]: Adjusting osd_memory_target on smithi062 to 7520M 2024-03-24T11:12:07.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:07 smithi139 ceph-mon[25719]: pgmap v51: 0 pgs: ; 0 B data, 4.8 MiB used, 89 GiB / 89 GiB avail 2024-03-24T11:12:07.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:07 smithi139 ceph-mon[25719]: from='osd.1 [v2:172.21.15.62:6810/1156951329,v1:172.21.15.62:6811/1156951329]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-24T11:12:07.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:07 smithi139 ceph-mon[25719]: osdmap e11: 2 total, 1 up, 2 in 2024-03-24T11:12:07.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:07 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T11:12:07.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:07 smithi139 ceph-mon[25719]: from='osd.1 [v2:172.21.15.62:6810/1156951329,v1:172.21.15.62:6811/1156951329]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:12:07.762 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-24T11:12:07.763 INFO:teuthology.orchestra.run.smithi062.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-24T11:12:07.763 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10+0 records in 2024-03-24T11:12:07.763 INFO:teuthology.orchestra.run.smithi062.stderr:10+0 records out 2024-03-24T11:12:07.763 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.012829 s, 817 MB/s 2024-03-24T11:12:07.763 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping successful for: 2024-03-24T11:12:08.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:08 smithi062 ceph-mon[24799]: from='osd.1 [v2:172.21.15.62:6810/1156951329,v1:172.21.15.62:6811/1156951329]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-03-24T11:12:08.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:08 smithi062 ceph-mon[24799]: osdmap e12: 2 total, 1 up, 2 in 2024-03-24T11:12:08.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:08 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T11:12:08.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:08 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T11:12:08.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:08 smithi062 ceph-mon[24799]: pgmap v54: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2024-03-24T11:12:08.343 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:08 smithi062 ceph-mon[29666]: from='osd.1 [v2:172.21.15.62:6810/1156951329,v1:172.21.15.62:6811/1156951329]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-03-24T11:12:08.343 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:08 smithi062 ceph-mon[29666]: osdmap e12: 2 total, 1 up, 2 in 2024-03-24T11:12:08.343 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:08 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T11:12:08.343 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:08 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T11:12:08.343 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:08 smithi062 ceph-mon[29666]: pgmap v54: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2024-03-24T11:12:08.367 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi062:vg_nvme/lv_2 2024-03-24T11:12:08.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:08 smithi139 ceph-mon[25719]: from='osd.1 [v2:172.21.15.62:6810/1156951329,v1:172.21.15.62:6811/1156951329]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-03-24T11:12:08.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:08 smithi139 ceph-mon[25719]: osdmap e12: 2 total, 1 up, 2 in 2024-03-24T11:12:08.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:08 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T11:12:08.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:08 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T11:12:08.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:08 smithi139 ceph-mon[25719]: pgmap v54: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2024-03-24T11:12:09.264 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:09 smithi062 ceph-mon[24799]: purged_snaps scrub starts 2024-03-24T11:12:09.264 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:09 smithi062 ceph-mon[24799]: purged_snaps scrub ok 2024-03-24T11:12:09.264 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:09 smithi062 ceph-mon[24799]: osd.1 [v2:172.21.15.62:6810/1156951329,v1:172.21.15.62:6811/1156951329] boot 2024-03-24T11:12:09.264 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:09 smithi062 ceph-mon[24799]: osdmap e13: 2 total, 2 up, 2 in 2024-03-24T11:12:09.265 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:09 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T11:12:09.265 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:09 smithi062 ceph-mon[29666]: purged_snaps scrub starts 2024-03-24T11:12:09.265 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:09 smithi062 ceph-mon[29666]: purged_snaps scrub ok 2024-03-24T11:12:09.265 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:09 smithi062 ceph-mon[29666]: osd.1 [v2:172.21.15.62:6810/1156951329,v1:172.21.15.62:6811/1156951329] boot 2024-03-24T11:12:09.265 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:09 smithi062 ceph-mon[29666]: osdmap e13: 2 total, 2 up, 2 in 2024-03-24T11:12:09.265 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:09 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T11:12:09.607 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:09 smithi139 ceph-mon[25719]: purged_snaps scrub starts 2024-03-24T11:12:09.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:09 smithi139 ceph-mon[25719]: purged_snaps scrub ok 2024-03-24T11:12:09.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:09 smithi139 ceph-mon[25719]: osd.1 [v2:172.21.15.62:6810/1156951329,v1:172.21.15.62:6811/1156951329] boot 2024-03-24T11:12:09.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:09 smithi139 ceph-mon[25719]: osdmap e13: 2 total, 2 up, 2 in 2024-03-24T11:12:09.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:09 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T11:12:10.402 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:10 smithi062 ceph-mon[24799]: osdmap e14: 2 total, 2 up, 2 in 2024-03-24T11:12:10.402 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:10 smithi062 ceph-mon[24799]: pgmap v57: 0 pgs: ; 0 B data, 9.7 MiB used, 179 GiB / 179 GiB avail 2024-03-24T11:12:10.403 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:10 smithi062 ceph-mon[29666]: osdmap e14: 2 total, 2 up, 2 in 2024-03-24T11:12:10.403 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:10 smithi062 ceph-mon[29666]: pgmap v57: 0 pgs: ; 0 B data, 9.7 MiB used, 179 GiB / 179 GiB avail 2024-03-24T11:12:10.607 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:10 smithi139 ceph-mon[25719]: osdmap e14: 2 total, 2 up, 2 in 2024-03-24T11:12:10.607 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:10 smithi139 ceph-mon[25719]: pgmap v57: 0 pgs: ; 0 B data, 9.7 MiB used, 179 GiB / 179 GiB avail 2024-03-24T11:12:10.976 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:12:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.62 - - [24/Mar/2024:11:12:10] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:12:11.307 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:11 smithi062 ceph-mon[24799]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:12:11.307 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:11 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T11:12:11.308 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:11 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T11:12:11.308 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:11 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:12:11.308 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:11 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:11.308 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:11 smithi062 ceph-mon[24799]: Detected new or changed devices on smithi062 2024-03-24T11:12:11.308 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:11 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:11.308 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:11 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:12:11.308 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:11 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:12:11.308 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:11 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:12:11.308 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:11 smithi062 ceph-mon[24799]: pgmap v58: 0 pgs: ; 0 B data, 9.7 MiB used, 179 GiB / 179 GiB avail 2024-03-24T11:12:11.308 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:11 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:11.308 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:11 smithi062 ceph-mon[29666]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:12:11.309 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:11 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T11:12:11.309 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:11 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T11:12:11.309 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:11 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:12:11.309 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:11 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:11.309 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:11 smithi062 ceph-mon[29666]: Detected new or changed devices on smithi062 2024-03-24T11:12:11.309 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:11 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:11.309 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:11 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:12:11.309 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:11 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:12:11.309 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:11 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:12:11.309 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:11 smithi062 ceph-mon[29666]: pgmap v58: 0 pgs: ; 0 B data, 9.7 MiB used, 179 GiB / 179 GiB avail 2024-03-24T11:12:11.309 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:11 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:11.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:11 smithi139 ceph-mon[25719]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:12:11.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:11 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T11:12:11.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:11 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T11:12:11.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:11 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:12:11.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:11 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:11.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:11 smithi139 ceph-mon[25719]: Detected new or changed devices on smithi062 2024-03-24T11:12:11.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:11 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:11.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:11 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:12:11.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:11 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:12:11.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:11 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:12:11.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:11 smithi139 ceph-mon[25719]: pgmap v58: 0 pgs: ; 0 B data, 9.7 MiB used, 179 GiB / 179 GiB avail 2024-03-24T11:12:11.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:11 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:12.405 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:12 smithi062 ceph-mon[24799]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-03-24T11:12:12.406 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:12 smithi062 ceph-mon[29666]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-03-24T11:12:12.607 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:12 smithi139 ceph-mon[25719]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-03-24T11:12:14.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:13 smithi139 ceph-mon[25719]: pgmap v59: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-03-24T11:12:14.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:13 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/2828010213' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "569ff950-c8b0-48fc-b137-4598571ff36c"}]: dispatch 2024-03-24T11:12:14.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:13 smithi139 ceph-mon[25719]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "569ff950-c8b0-48fc-b137-4598571ff36c"}]: dispatch 2024-03-24T11:12:14.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:13 smithi139 ceph-mon[25719]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "569ff950-c8b0-48fc-b137-4598571ff36c"}]': finished 2024-03-24T11:12:14.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:13 smithi139 ceph-mon[25719]: osdmap e15: 3 total, 2 up, 3 in 2024-03-24T11:12:14.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:13 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T11:12:14.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:13 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/2022975581' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T11:12:14.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:13 smithi062 ceph-mon[24799]: pgmap v59: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-03-24T11:12:14.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:13 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/2828010213' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "569ff950-c8b0-48fc-b137-4598571ff36c"}]: dispatch 2024-03-24T11:12:14.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:13 smithi062 ceph-mon[24799]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "569ff950-c8b0-48fc-b137-4598571ff36c"}]: dispatch 2024-03-24T11:12:14.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:13 smithi062 ceph-mon[24799]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "569ff950-c8b0-48fc-b137-4598571ff36c"}]': finished 2024-03-24T11:12:14.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:13 smithi062 ceph-mon[24799]: osdmap e15: 3 total, 2 up, 3 in 2024-03-24T11:12:14.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:13 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T11:12:14.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:13 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/2022975581' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T11:12:14.228 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:12:13 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.62 - - [24/Mar/2024:11:12:13] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:12:14.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:13 smithi062 ceph-mon[29666]: pgmap v59: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-03-24T11:12:14.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:13 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/2828010213' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "569ff950-c8b0-48fc-b137-4598571ff36c"}]: dispatch 2024-03-24T11:12:14.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:13 smithi062 ceph-mon[29666]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "569ff950-c8b0-48fc-b137-4598571ff36c"}]: dispatch 2024-03-24T11:12:14.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:13 smithi062 ceph-mon[29666]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "569ff950-c8b0-48fc-b137-4598571ff36c"}]': finished 2024-03-24T11:12:14.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:13 smithi062 ceph-mon[29666]: osdmap e15: 3 total, 2 up, 3 in 2024-03-24T11:12:14.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:13 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T11:12:14.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:13 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/2022975581' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T11:12:15.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:14 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:15.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:14 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:15.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:14 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:12:15.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:14 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:12:15.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:14 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:12:15.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:14 smithi139 ceph-mon[25719]: pgmap v61: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-03-24T11:12:15.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:14 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:15.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:14 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:15.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:14 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:15.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:14 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:12:15.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:14 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:12:15.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:14 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:12:15.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:14 smithi062 ceph-mon[24799]: pgmap v61: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-03-24T11:12:15.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:14 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:15.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:14 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:15.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:14 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:15.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:14 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:12:15.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:14 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:12:15.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:14 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:12:15.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:14 smithi062 ceph-mon[29666]: pgmap v61: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-03-24T11:12:15.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:14 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:16.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:15 smithi139 ceph-mon[25719]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-03-24T11:12:16.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:15 smithi139 ceph-mon[25719]: Cluster is now healthy 2024-03-24T11:12:16.208 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:15 smithi062 ceph-mon[24799]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-03-24T11:12:16.208 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:15 smithi062 ceph-mon[24799]: Cluster is now healthy 2024-03-24T11:12:16.208 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:15 smithi062 ceph-mon[29666]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-03-24T11:12:16.208 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:15 smithi062 ceph-mon[29666]: Cluster is now healthy 2024-03-24T11:12:17.064 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:16 smithi062 ceph-mon[24799]: pgmap v62: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-03-24T11:12:17.064 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:16 smithi062 ceph-mon[29666]: pgmap v62: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-03-24T11:12:17.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:16 smithi139 ceph-mon[25719]: pgmap v62: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-03-24T11:12:18.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:18 smithi062 ceph-mon[24799]: pgmap v63: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-03-24T11:12:18.839 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:18 smithi062 ceph-mon[29666]: pgmap v63: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-03-24T11:12:19.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:18 smithi139 ceph-mon[25719]: pgmap v63: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-03-24T11:12:21.218 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:20 smithi062 ceph-mon[24799]: pgmap v64: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-03-24T11:12:21.219 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:20 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-24T11:12:21.219 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:20 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:12:21.219 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:20 smithi062 ceph-mon[24799]: Deploying daemon osd.2 on smithi062 2024-03-24T11:12:21.219 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:20 smithi062 ceph-mon[29666]: pgmap v64: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-03-24T11:12:21.219 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:20 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-24T11:12:21.219 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:20 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:12:21.220 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:20 smithi062 ceph-mon[29666]: Deploying daemon osd.2 on smithi062 2024-03-24T11:12:21.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:20 smithi139 ceph-mon[25719]: pgmap v64: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-03-24T11:12:21.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:20 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-24T11:12:21.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:20 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:12:21.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:20 smithi139 ceph-mon[25719]: Deploying daemon osd.2 on smithi062 2024-03-24T11:12:23.178 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:22 smithi062 ceph-mon[24799]: pgmap v65: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-03-24T11:12:23.179 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:22 smithi062 ceph-mon[29666]: pgmap v65: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-03-24T11:12:23.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:22 smithi139 ceph-mon[25719]: pgmap v65: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-03-24T11:12:23.727 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:12:23 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.139 - - [24/Mar/2024:11:12:23] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:12:24.726 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:24 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:24.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:24 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:24.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:24 smithi062 ceph-mon[24799]: pgmap v66: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-03-24T11:12:24.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:24 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-24T11:12:24.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:24 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-24T11:12:24.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:24 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:24.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:24 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:24.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:24 smithi062 ceph-mon[29666]: pgmap v66: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-03-24T11:12:24.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:24 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-24T11:12:24.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:24 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-24T11:12:24.821 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:24 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:24.821 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:24 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:24.821 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:24 smithi139 ceph-mon[25719]: pgmap v66: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-03-24T11:12:24.821 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:24 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-24T11:12:24.821 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:24 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-24T11:12:25.958 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:25 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:25.958 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:25 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:12:25.958 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:25 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:12:25.958 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:25 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:12:25.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:25 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:25.959 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:25 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:25.959 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:25 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:12:25.959 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:25 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:12:25.959 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:25 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:12:25.959 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:25 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:26.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:25 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:26.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:25 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:12:26.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:25 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:12:26.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:25 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:12:26.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:25 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:26.817 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:26 smithi139 ceph-mon[25719]: pgmap v67: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-03-24T11:12:26.976 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:26 smithi062 ceph-mon[24799]: pgmap v67: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-03-24T11:12:26.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:26 smithi062 ceph-mon[29666]: pgmap v67: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-03-24T11:12:27.364 INFO:teuthology.orchestra.run.smithi062.stdout:Created osd(s) 2 on host 'smithi062' 2024-03-24T11:12:28.010 DEBUG:teuthology.orchestra.run.smithi062:osd.2> sudo journalctl -f -n 0 -u ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.2.service 2024-03-24T11:12:28.013 INFO:tasks.cephadm:Deploying osd.3 on smithi062 with /dev/vg_nvme/lv_1... 2024-03-24T11:12:28.013 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-24T11:12:28.655 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:28 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:28.655 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:28 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:12:28.655 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:28 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:12:28.655 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:28 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:12:28.655 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:28 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:12:28.655 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:28 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:12:28.655 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:28 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:12:28.655 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:28 smithi062 ceph-mon[29666]: Adjusting osd_memory_target on smithi062 to 5013M 2024-03-24T11:12:28.655 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:28 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:28.655 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:28 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:28.655 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:28 smithi062 ceph-mon[29666]: from='osd.2 [v2:172.21.15.62:6818/2474659851,v1:172.21.15.62:6819/2474659851]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-24T11:12:28.655 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:28 smithi062 ceph-mon[29666]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-24T11:12:28.656 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:28 smithi062 ceph-mon[29666]: pgmap v68: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-03-24T11:12:28.656 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:28 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:28.656 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:28 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:12:28.656 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:28 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:12:28.656 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:28 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:12:28.656 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:28 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:12:28.656 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:28 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:12:28.656 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:28 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:12:28.656 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:28 smithi062 ceph-mon[24799]: Adjusting osd_memory_target on smithi062 to 5013M 2024-03-24T11:12:28.656 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:28 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:28.656 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:28 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:28.656 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:28 smithi062 ceph-mon[24799]: from='osd.2 [v2:172.21.15.62:6818/2474659851,v1:172.21.15.62:6819/2474659851]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-24T11:12:28.656 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:28 smithi062 ceph-mon[24799]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-24T11:12:28.657 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:28 smithi062 ceph-mon[24799]: pgmap v68: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-03-24T11:12:28.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:28 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:28.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:28 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:12:28.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:28 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:12:28.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:28 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:12:28.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:28 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:12:28.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:28 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:12:28.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:28 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:12:28.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:28 smithi139 ceph-mon[25719]: Adjusting osd_memory_target on smithi062 to 5013M 2024-03-24T11:12:28.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:28 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:28.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:28 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:28.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:28 smithi139 ceph-mon[25719]: from='osd.2 [v2:172.21.15.62:6818/2474659851,v1:172.21.15.62:6819/2474659851]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-24T11:12:28.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:28 smithi139 ceph-mon[25719]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-24T11:12:28.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:28 smithi139 ceph-mon[25719]: pgmap v68: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-03-24T11:12:29.551 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:29 smithi062 ceph-mon[24799]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-24T11:12:29.552 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:29 smithi062 ceph-mon[24799]: osdmap e16: 3 total, 2 up, 3 in 2024-03-24T11:12:29.552 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:29 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T11:12:29.552 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:29 smithi062 ceph-mon[24799]: from='osd.2 [v2:172.21.15.62:6818/2474659851,v1:172.21.15.62:6819/2474659851]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:12:29.552 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:29 smithi062 ceph-mon[24799]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:12:29.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:29 smithi062 ceph-mon[29666]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-24T11:12:29.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:29 smithi062 ceph-mon[29666]: osdmap e16: 3 total, 2 up, 3 in 2024-03-24T11:12:29.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:29 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T11:12:29.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:29 smithi062 ceph-mon[29666]: from='osd.2 [v2:172.21.15.62:6818/2474659851,v1:172.21.15.62:6819/2474659851]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:12:29.552 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:29 smithi062 ceph-mon[29666]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:12:29.553 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:12:29 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2[38778]: 2024-03-24T11:12:29.413+0000 7f7e56cf7700 -1 osd.2 0 waiting for initial osdmap 2024-03-24T11:12:29.553 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:12:29 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2[38778]: 2024-03-24T11:12:29.416+0000 7f7e52690700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T11:12:29.857 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:29 smithi139 ceph-mon[25719]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-24T11:12:29.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:29 smithi139 ceph-mon[25719]: osdmap e16: 3 total, 2 up, 3 in 2024-03-24T11:12:29.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:29 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T11:12:29.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:29 smithi139 ceph-mon[25719]: from='osd.2 [v2:172.21.15.62:6818/2474659851,v1:172.21.15.62:6819/2474659851]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:12:29.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:29 smithi139 ceph-mon[25719]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:12:30.016 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-24T11:12:30.016 INFO:teuthology.orchestra.run.smithi062.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-24T11:12:30.017 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10+0 records in 2024-03-24T11:12:30.017 INFO:teuthology.orchestra.run.smithi062.stderr:10+0 records out 2024-03-24T11:12:30.017 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0146316 s, 717 MB/s 2024-03-24T11:12:30.017 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping successful for: 2024-03-24T11:12:30.571 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi062:vg_nvme/lv_1 2024-03-24T11:12:30.726 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:30 smithi062 ceph-mon[24799]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-03-24T11:12:30.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:30 smithi062 ceph-mon[24799]: osdmap e17: 3 total, 2 up, 3 in 2024-03-24T11:12:30.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:30 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T11:12:30.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:30 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T11:12:30.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:30 smithi062 ceph-mon[24799]: pgmap v71: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-03-24T11:12:30.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:30 smithi062 ceph-mon[29666]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-03-24T11:12:30.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:30 smithi062 ceph-mon[29666]: osdmap e17: 3 total, 2 up, 3 in 2024-03-24T11:12:30.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:30 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T11:12:30.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:30 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T11:12:30.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:30 smithi062 ceph-mon[29666]: pgmap v71: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-03-24T11:12:30.857 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:30 smithi139 ceph-mon[25719]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-03-24T11:12:30.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:30 smithi139 ceph-mon[25719]: osdmap e17: 3 total, 2 up, 3 in 2024-03-24T11:12:30.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:30 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T11:12:30.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:30 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T11:12:30.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:30 smithi139 ceph-mon[25719]: pgmap v71: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2024-03-24T11:12:31.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:31 smithi062 ceph-mon[24799]: purged_snaps scrub starts 2024-03-24T11:12:31.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:31 smithi062 ceph-mon[24799]: purged_snaps scrub ok 2024-03-24T11:12:31.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:31 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T11:12:31.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:31 smithi062 ceph-mon[24799]: osd.2 [v2:172.21.15.62:6818/2474659851,v1:172.21.15.62:6819/2474659851] boot 2024-03-24T11:12:31.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:31 smithi062 ceph-mon[24799]: osdmap e18: 3 total, 3 up, 3 in 2024-03-24T11:12:31.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:31 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T11:12:31.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:31 smithi062 ceph-mon[29666]: purged_snaps scrub starts 2024-03-24T11:12:31.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:31 smithi062 ceph-mon[29666]: purged_snaps scrub ok 2024-03-24T11:12:31.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:31 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T11:12:31.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:31 smithi062 ceph-mon[29666]: osd.2 [v2:172.21.15.62:6818/2474659851,v1:172.21.15.62:6819/2474659851] boot 2024-03-24T11:12:31.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:31 smithi062 ceph-mon[29666]: osdmap e18: 3 total, 3 up, 3 in 2024-03-24T11:12:31.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:31 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T11:12:31.857 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:31 smithi139 ceph-mon[25719]: purged_snaps scrub starts 2024-03-24T11:12:31.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:31 smithi139 ceph-mon[25719]: purged_snaps scrub ok 2024-03-24T11:12:31.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:31 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T11:12:31.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:31 smithi139 ceph-mon[25719]: osd.2 [v2:172.21.15.62:6818/2474659851,v1:172.21.15.62:6819/2474659851] boot 2024-03-24T11:12:31.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:31 smithi139 ceph-mon[25719]: osdmap e18: 3 total, 3 up, 3 in 2024-03-24T11:12:31.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:31 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T11:12:32.693 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:32 smithi062 ceph-mon[24799]: osdmap e19: 3 total, 3 up, 3 in 2024-03-24T11:12:32.693 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:32 smithi062 ceph-mon[24799]: pgmap v74: 0 pgs: ; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-03-24T11:12:32.693 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:32 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-03-24T11:12:32.693 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:32 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T11:12:32.693 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:32 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T11:12:32.693 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:32 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:12:32.694 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:32 smithi062 ceph-mon[29666]: osdmap e19: 3 total, 3 up, 3 in 2024-03-24T11:12:32.694 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:32 smithi062 ceph-mon[29666]: pgmap v74: 0 pgs: ; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-03-24T11:12:32.694 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:32 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-03-24T11:12:32.694 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:32 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T11:12:32.694 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:32 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T11:12:32.694 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:32 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:12:32.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:32 smithi139 ceph-mon[25719]: osdmap e19: 3 total, 3 up, 3 in 2024-03-24T11:12:32.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:32 smithi139 ceph-mon[25719]: pgmap v74: 0 pgs: ; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-03-24T11:12:32.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:32 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-03-24T11:12:32.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:32 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T11:12:32.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:32 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T11:12:32.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:32 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:12:33.726 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:33 smithi062 ceph-mon[24799]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:12:33.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:33 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-03-24T11:12:33.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:33 smithi062 ceph-mon[24799]: osdmap e20: 3 total, 3 up, 3 in 2024-03-24T11:12:33.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:33 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-24T11:12:33.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:33 smithi062 ceph-mon[29666]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:12:33.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:33 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-03-24T11:12:33.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:33 smithi062 ceph-mon[29666]: osdmap e20: 3 total, 3 up, 3 in 2024-03-24T11:12:33.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:33 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-24T11:12:33.759 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:33 smithi139 ceph-mon[25719]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:12:33.759 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:33 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-03-24T11:12:33.759 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:33 smithi139 ceph-mon[25719]: osdmap e20: 3 total, 3 up, 3 in 2024-03-24T11:12:33.759 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:33 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-24T11:12:34.084 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:12:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.62 - - [24/Mar/2024:11:12:33] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:12:34.084 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:12:33 smithi062 sudo[40006]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-24T11:12:34.084 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:12:33 smithi062 sudo[40006]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-03-24T11:12:34.084 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:12:33 smithi062 sudo[40006]: pam_unix(sudo:session): session closed for user root 2024-03-24T11:12:34.336 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:12:34 smithi062 sudo[40057]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-24T11:12:34.336 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:12:34 smithi062 sudo[40057]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-03-24T11:12:34.337 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:12:34 smithi062 sudo[40057]: pam_unix(sudo:session): session closed for user root 2024-03-24T11:12:34.656 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:12:34 smithi062 sudo[40065]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-24T11:12:34.656 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:12:34 smithi062 sudo[40065]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-03-24T11:12:34.656 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:12:34 smithi062 sudo[40065]: pam_unix(sudo:session): session closed for user root 2024-03-24T11:12:34.657 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:34 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-24T11:12:34.657 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:34 smithi062 ceph-mon[24799]: osdmap e21: 3 total, 3 up, 3 in 2024-03-24T11:12:34.657 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:34 smithi062 ceph-mon[24799]: pgmap v77: 1 pgs: 1 creating+peering; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-03-24T11:12:34.657 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:34 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:34.657 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:34 smithi062 ceph-mon[24799]: Detected new or changed devices on smithi062 2024-03-24T11:12:34.657 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:34 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:34.657 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:34 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:12:34.657 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:34 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:12:34.657 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:34 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:12:34.657 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:34 smithi062 ceph-mon[24799]: pgmap v78: 1 pgs: 1 creating+peering; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-03-24T11:12:34.657 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:34 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:34.657 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:34 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:34.658 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:34 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-24T11:12:34.658 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:34 smithi062 ceph-mon[29666]: osdmap e21: 3 total, 3 up, 3 in 2024-03-24T11:12:34.658 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:34 smithi062 ceph-mon[29666]: pgmap v77: 1 pgs: 1 creating+peering; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-03-24T11:12:34.658 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:34 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:34.658 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:34 smithi062 ceph-mon[29666]: Detected new or changed devices on smithi062 2024-03-24T11:12:34.658 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:34 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:34.658 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:34 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:12:34.658 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:34 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:12:34.658 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:34 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:12:34.658 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:34 smithi062 ceph-mon[29666]: pgmap v78: 1 pgs: 1 creating+peering; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-03-24T11:12:34.659 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:34 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:34.659 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:34 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:34.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:34 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-24T11:12:34.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:34 smithi139 ceph-mon[25719]: osdmap e21: 3 total, 3 up, 3 in 2024-03-24T11:12:34.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:34 smithi139 ceph-mon[25719]: pgmap v77: 1 pgs: 1 creating+peering; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-03-24T11:12:34.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:34 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:34.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:34 smithi139 ceph-mon[25719]: Detected new or changed devices on smithi062 2024-03-24T11:12:34.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:34 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:34.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:34 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:12:34.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:34 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:12:34.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:34 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:12:34.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:34 smithi139 ceph-mon[25719]: pgmap v78: 1 pgs: 1 creating+peering; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-03-24T11:12:34.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:34 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:34.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:34 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:34.971 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:12:34 smithi062 sudo[40089]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-24T11:12:34.971 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:12:34 smithi062 sudo[40089]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-03-24T11:12:34.971 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:12:34 smithi062 sudo[40089]: pam_unix(sudo:session): session closed for user root 2024-03-24T11:12:35.226 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:12:34 smithi062 sudo[40092]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-24T11:12:35.227 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:12:34 smithi062 sudo[40092]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-03-24T11:12:35.227 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:12:34 smithi062 sudo[40092]: pam_unix(sudo:session): session closed for user root 2024-03-24T11:12:35.503 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:12:35 smithi062 sudo[40132]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-24T11:12:35.503 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:12:35 smithi062 sudo[40132]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-03-24T11:12:35.503 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:12:35 smithi062 sudo[40132]: pam_unix(sudo:session): session closed for user root 2024-03-24T11:12:35.777 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:35 smithi062 sudo[40148]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-24T11:12:35.777 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:35 smithi062 sudo[40148]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-03-24T11:12:35.777 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:35 smithi062 sudo[40148]: pam_unix(sudo:session): session closed for user root 2024-03-24T11:12:36.040 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:35 smithi062 ceph-mon[29666]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-03-24T11:12:36.041 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:35 smithi062 ceph-mon[29666]: osdmap e22: 3 total, 3 up, 3 in 2024-03-24T11:12:36.041 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:35 smithi062 ceph-mon[29666]: mgrmap e15: y(active, since 2m), standbys: x 2024-03-24T11:12:36.041 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:35 smithi062 ceph-mon[29666]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-24T11:12:36.041 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:35 smithi062 sudo[40233]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-24T11:12:36.041 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:35 smithi062 sudo[40233]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-03-24T11:12:36.041 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:35 smithi062 sudo[40233]: pam_unix(sudo:session): session closed for user root 2024-03-24T11:12:36.041 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:35 smithi062 ceph-mon[24799]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-03-24T11:12:36.041 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:35 smithi062 ceph-mon[24799]: osdmap e22: 3 total, 3 up, 3 in 2024-03-24T11:12:36.041 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:35 smithi062 ceph-mon[24799]: mgrmap e15: y(active, since 2m), standbys: x 2024-03-24T11:12:36.041 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:35 smithi062 ceph-mon[24799]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-24T11:12:36.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:35 smithi139 ceph-mon[25719]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-03-24T11:12:36.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:35 smithi139 ceph-mon[25719]: osdmap e22: 3 total, 3 up, 3 in 2024-03-24T11:12:36.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:35 smithi139 ceph-mon[25719]: mgrmap e15: y(active, since 2m), standbys: x 2024-03-24T11:12:36.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:35 smithi139 ceph-mon[25719]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-24T11:12:36.307 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:36 smithi062 sudo[40236]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-24T11:12:36.307 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:36 smithi062 sudo[40236]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-03-24T11:12:36.307 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:36 smithi062 sudo[40236]: pam_unix(sudo:session): session closed for user root 2024-03-24T11:12:36.726 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:36 smithi062 sudo[40239]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-24T11:12:36.726 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:36 smithi062 sudo[40239]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-03-24T11:12:36.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:36 smithi062 sudo[40239]: pam_unix(sudo:session): session closed for user root 2024-03-24T11:12:36.802 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:36 smithi139 sudo[28200]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-24T11:12:36.802 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:36 smithi139 sudo[28200]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-03-24T11:12:36.802 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:36 smithi139 sudo[28200]: pam_unix(sudo:session): session closed for user root 2024-03-24T11:12:37.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:36 smithi139 sudo[28203]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-24T11:12:37.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:36 smithi139 sudo[28203]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-03-24T11:12:37.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:36 smithi139 sudo[28203]: pam_unix(sudo:session): session closed for user root 2024-03-24T11:12:37.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:36 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/3787520690' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0c09510-dcba-4332-9c02-3842c992612b"}]: dispatch 2024-03-24T11:12:37.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:36 smithi139 ceph-mon[25719]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-24T11:12:37.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:36 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T11:12:37.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:36 smithi139 ceph-mon[25719]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0c09510-dcba-4332-9c02-3842c992612b"}]: dispatch 2024-03-24T11:12:37.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:36 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T11:12:37.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:36 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:12:37.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:36 smithi139 ceph-mon[25719]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-24T11:12:37.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:36 smithi139 ceph-mon[25719]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e0c09510-dcba-4332-9c02-3842c992612b"}]': finished 2024-03-24T11:12:37.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:36 smithi139 ceph-mon[25719]: osdmap e23: 4 total, 3 up, 4 in 2024-03-24T11:12:37.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:36 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T11:12:37.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:36 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T11:12:37.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:36 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:12:37.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:36 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T11:12:37.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:36 smithi139 ceph-mon[25719]: pgmap v81: 1 pgs: 1 creating+peering; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-03-24T11:12:37.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:36 smithi139 ceph-mon[25719]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-24T11:12:37.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:36 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T11:12:37.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:36 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T11:12:37.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:36 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:12:37.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:36 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:37.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:36 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:12:37.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:36 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:12:37.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:36 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:12:37.226 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:12:36 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.62 - - [24/Mar/2024:11:12:36] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:12:37.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:36 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/3787520690' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0c09510-dcba-4332-9c02-3842c992612b"}]: dispatch 2024-03-24T11:12:37.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:36 smithi062 ceph-mon[24799]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-24T11:12:37.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:36 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T11:12:37.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:36 smithi062 ceph-mon[24799]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0c09510-dcba-4332-9c02-3842c992612b"}]: dispatch 2024-03-24T11:12:37.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:36 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T11:12:37.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:36 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:12:37.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:36 smithi062 ceph-mon[24799]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-24T11:12:37.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:36 smithi062 ceph-mon[24799]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e0c09510-dcba-4332-9c02-3842c992612b"}]': finished 2024-03-24T11:12:37.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:36 smithi062 ceph-mon[24799]: osdmap e23: 4 total, 3 up, 4 in 2024-03-24T11:12:37.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:36 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T11:12:37.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:36 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T11:12:37.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:36 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:12:37.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:36 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T11:12:37.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:36 smithi062 ceph-mon[24799]: pgmap v81: 1 pgs: 1 creating+peering; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-03-24T11:12:37.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:36 smithi062 ceph-mon[24799]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-24T11:12:37.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:36 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T11:12:37.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:36 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T11:12:37.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:36 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:12:37.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:36 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:37.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:36 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:12:37.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:36 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:12:37.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:36 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:12:37.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:36 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/3787520690' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0c09510-dcba-4332-9c02-3842c992612b"}]: dispatch 2024-03-24T11:12:37.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:36 smithi062 ceph-mon[29666]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-24T11:12:37.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:36 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T11:12:37.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:36 smithi062 ceph-mon[29666]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0c09510-dcba-4332-9c02-3842c992612b"}]: dispatch 2024-03-24T11:12:37.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:36 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T11:12:37.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:36 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:12:37.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:36 smithi062 ceph-mon[29666]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-24T11:12:37.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:36 smithi062 ceph-mon[29666]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e0c09510-dcba-4332-9c02-3842c992612b"}]': finished 2024-03-24T11:12:37.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:36 smithi062 ceph-mon[29666]: osdmap e23: 4 total, 3 up, 4 in 2024-03-24T11:12:37.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:36 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T11:12:37.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:36 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T11:12:37.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:36 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:12:37.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:36 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T11:12:37.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:36 smithi062 ceph-mon[29666]: pgmap v81: 1 pgs: 1 creating+peering; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-03-24T11:12:37.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:36 smithi062 ceph-mon[29666]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-24T11:12:37.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:36 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T11:12:37.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:36 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T11:12:37.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:36 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:12:37.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:36 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:37.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:36 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:12:37.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:36 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:12:37.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:36 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:12:38.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:37 smithi139 ceph-mon[25719]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-24T11:12:38.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:37 smithi139 ceph-mon[25719]: pgmap v82: 1 pgs: 1 creating+peering; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-03-24T11:12:38.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:37 smithi139 ceph-mon[25719]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-03-24T11:12:38.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:37 smithi139 ceph-mon[25719]: Cluster is now healthy 2024-03-24T11:12:38.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:37 smithi139 ceph-mon[25719]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-24T11:12:38.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:37 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:38.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:37 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:38.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:37 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:38.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:37 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/1802977599' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T11:12:38.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:37 smithi062 ceph-mon[24799]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-24T11:12:38.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:37 smithi062 ceph-mon[24799]: pgmap v82: 1 pgs: 1 creating+peering; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-03-24T11:12:38.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:37 smithi062 ceph-mon[24799]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-03-24T11:12:38.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:37 smithi062 ceph-mon[24799]: Cluster is now healthy 2024-03-24T11:12:38.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:37 smithi062 ceph-mon[24799]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-24T11:12:38.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:37 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:38.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:37 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:38.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:37 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:38.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:37 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/1802977599' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T11:12:38.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:37 smithi062 ceph-mon[29666]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-24T11:12:38.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:37 smithi062 ceph-mon[29666]: pgmap v82: 1 pgs: 1 creating+peering; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-03-24T11:12:38.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:37 smithi062 ceph-mon[29666]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-03-24T11:12:38.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:37 smithi062 ceph-mon[29666]: Cluster is now healthy 2024-03-24T11:12:38.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:37 smithi062 ceph-mon[29666]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-24T11:12:38.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:37 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:38.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:37 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:38.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:37 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:38.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:37 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/1802977599' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T11:12:39.976 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:39 smithi062 ceph-mon[24799]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2024-03-24T11:12:39.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:39 smithi062 ceph-mon[29666]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2024-03-24T11:12:40.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:39 smithi139 ceph-mon[25719]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2024-03-24T11:12:42.098 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:41 smithi062 ceph-mon[24799]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2024-03-24T11:12:42.098 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:41 smithi062 ceph-mon[29666]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2024-03-24T11:12:42.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:41 smithi139 ceph-mon[25719]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2024-03-24T11:12:43.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:43 smithi139 ceph-mon[25719]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2024-03-24T11:12:43.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:43 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-24T11:12:43.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:43 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:12:43.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:43 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:43.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:43 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:43.900 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:12:43 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.139 - - [24/Mar/2024:11:12:43] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:12:43.900 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:43 smithi062 ceph-mon[29666]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2024-03-24T11:12:43.900 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:43 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-24T11:12:43.900 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:43 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:12:43.900 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:43 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:43.900 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:43 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:43.901 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:43 smithi062 ceph-mon[24799]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2024-03-24T11:12:43.901 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:43 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-24T11:12:43.901 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:43 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:12:43.901 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:43 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:43.901 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:43 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:45.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:44 smithi139 ceph-mon[25719]: Deploying daemon osd.3 on smithi062 2024-03-24T11:12:45.134 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:44 smithi062 ceph-mon[24799]: Deploying daemon osd.3 on smithi062 2024-03-24T11:12:45.134 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:44 smithi062 ceph-mon[29666]: Deploying daemon osd.3 on smithi062 2024-03-24T11:12:45.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:45 smithi062 ceph-mon[29666]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2024-03-24T11:12:45.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:45 smithi062 ceph-mon[24799]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2024-03-24T11:12:46.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:45 smithi139 ceph-mon[25719]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2024-03-24T11:12:48.107 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:47 smithi062 ceph-mon[29666]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2024-03-24T11:12:48.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:47 smithi062 ceph-mon[24799]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2024-03-24T11:12:48.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:47 smithi139 ceph-mon[25719]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2024-03-24T11:12:49.300 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:49 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:49.300 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:49 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:12:49.300 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:49 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:12:49.300 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:49 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:12:49.301 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:49 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:49.301 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:49 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:49.301 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:49 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:12:49.301 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:49 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:12:49.301 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:49 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:12:49.301 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:49 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:49.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:49 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:49.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:49 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:12:49.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:49 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:12:49.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:49 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:12:49.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:49 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:50.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:50 smithi062 ceph-mon[24799]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2024-03-24T11:12:50.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:50 smithi062 ceph-mon[29666]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2024-03-24T11:12:50.607 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:50 smithi139 ceph-mon[25719]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2024-03-24T11:12:50.769 INFO:teuthology.orchestra.run.smithi062.stdout:Created osd(s) 3 on host 'smithi062' 2024-03-24T11:12:51.392 DEBUG:teuthology.orchestra.run.smithi062:osd.3> sudo journalctl -f -n 0 -u ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.3.service 2024-03-24T11:12:51.395 INFO:tasks.cephadm:Deploying osd.4 on smithi139 with /dev/vg_nvme/lv_4... 2024-03-24T11:12:51.395 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-24T11:12:52.038 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:51 smithi139 ceph-mon[25719]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2024-03-24T11:12:52.038 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:51 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:52.038 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:51 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:12:52.038 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:51 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:12:52.038 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:51 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:12:52.038 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:51 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:12:52.038 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:51 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:12:52.038 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:51 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:12:52.038 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:51 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:12:52.038 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:51 smithi139 ceph-mon[25719]: Adjusting osd_memory_target on smithi062 to 3760M 2024-03-24T11:12:52.038 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:51 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:52.039 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:51 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:52.039 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:51 smithi139 ceph-mon[25719]: from='osd.3 [v2:172.21.15.62:6826/2104856662,v1:172.21.15.62:6827/2104856662]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-24T11:12:52.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:51 smithi062 ceph-mon[24799]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2024-03-24T11:12:52.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:51 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:52.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:51 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:12:52.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:51 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:12:52.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:51 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:12:52.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:51 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:12:52.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:51 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:12:52.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:51 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:12:52.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:51 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:12:52.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:51 smithi062 ceph-mon[24799]: Adjusting osd_memory_target on smithi062 to 3760M 2024-03-24T11:12:52.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:51 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:52.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:51 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:52.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:51 smithi062 ceph-mon[24799]: from='osd.3 [v2:172.21.15.62:6826/2104856662,v1:172.21.15.62:6827/2104856662]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-24T11:12:52.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:51 smithi062 ceph-mon[29666]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2024-03-24T11:12:52.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:51 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:52.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:51 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:12:52.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:51 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:12:52.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:51 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:12:52.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:51 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:12:52.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:51 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:12:52.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:51 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:12:52.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:51 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:12:52.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:51 smithi062 ceph-mon[29666]: Adjusting osd_memory_target on smithi062 to 3760M 2024-03-24T11:12:52.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:51 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:52.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:51 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:52.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:51 smithi062 ceph-mon[29666]: from='osd.3 [v2:172.21.15.62:6826/2104856662,v1:172.21.15.62:6827/2104856662]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-24T11:12:53.188 INFO:teuthology.orchestra.run.smithi139.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-24T11:12:53.188 INFO:teuthology.orchestra.run.smithi139.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-24T11:12:53.188 INFO:teuthology.orchestra.run.smithi139.stderr: stderr: 10+0 records in 2024-03-24T11:12:53.188 INFO:teuthology.orchestra.run.smithi139.stderr:10+0 records out 2024-03-24T11:12:53.188 INFO:teuthology.orchestra.run.smithi139.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0131344 s, 798 MB/s 2024-03-24T11:12:53.188 INFO:teuthology.orchestra.run.smithi139.stderr:--> Zapping successful for: 2024-03-24T11:12:53.476 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:53 smithi062 ceph-mon[29666]: from='osd.3 [v2:172.21.15.62:6826/2104856662,v1:172.21.15.62:6827/2104856662]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-24T11:12:53.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:53 smithi062 ceph-mon[29666]: osdmap e24: 4 total, 3 up, 4 in 2024-03-24T11:12:53.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:53 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T11:12:53.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:53 smithi062 ceph-mon[29666]: from='osd.3 [v2:172.21.15.62:6826/2104856662,v1:172.21.15.62:6827/2104856662]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:12:53.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:53 smithi062 ceph-mon[24799]: from='osd.3 [v2:172.21.15.62:6826/2104856662,v1:172.21.15.62:6827/2104856662]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-24T11:12:53.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:53 smithi062 ceph-mon[24799]: osdmap e24: 4 total, 3 up, 4 in 2024-03-24T11:12:53.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:53 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T11:12:53.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:53 smithi062 ceph-mon[24799]: from='osd.3 [v2:172.21.15.62:6826/2104856662,v1:172.21.15.62:6827/2104856662]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:12:53.477 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:12:53 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3[41767]: 2024-03-24T11:12:53.123+0000 7f98b1f58700 -1 osd.3 0 waiting for initial osdmap 2024-03-24T11:12:53.478 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:12:53 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3[41767]: 2024-03-24T11:12:53.129+0000 7f98ac8ef700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T11:12:53.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:53 smithi139 ceph-mon[25719]: from='osd.3 [v2:172.21.15.62:6826/2104856662,v1:172.21.15.62:6827/2104856662]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-24T11:12:53.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:53 smithi139 ceph-mon[25719]: osdmap e24: 4 total, 3 up, 4 in 2024-03-24T11:12:53.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:53 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T11:12:53.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:53 smithi139 ceph-mon[25719]: from='osd.3 [v2:172.21.15.62:6826/2104856662,v1:172.21.15.62:6827/2104856662]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:12:53.786 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi139:vg_nvme/lv_4 2024-03-24T11:12:54.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:54 smithi139 ceph-mon[25719]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2024-03-24T11:12:54.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:54 smithi139 ceph-mon[25719]: from='osd.3 [v2:172.21.15.62:6826/2104856662,v1:172.21.15.62:6827/2104856662]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-03-24T11:12:54.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:54 smithi139 ceph-mon[25719]: osdmap e25: 4 total, 3 up, 4 in 2024-03-24T11:12:54.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:54 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T11:12:54.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:54 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T11:12:54.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:54 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:54.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:54 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:54.477 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:12:54 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.62 - - [24/Mar/2024:11:12:54] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:12:54.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:54 smithi062 ceph-mon[24799]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2024-03-24T11:12:54.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:54 smithi062 ceph-mon[24799]: from='osd.3 [v2:172.21.15.62:6826/2104856662,v1:172.21.15.62:6827/2104856662]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-03-24T11:12:54.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:54 smithi062 ceph-mon[24799]: osdmap e25: 4 total, 3 up, 4 in 2024-03-24T11:12:54.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:54 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T11:12:54.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:54 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T11:12:54.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:54 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:54.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:54 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:54.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:54 smithi062 ceph-mon[29666]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2024-03-24T11:12:54.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:54 smithi062 ceph-mon[29666]: from='osd.3 [v2:172.21.15.62:6826/2104856662,v1:172.21.15.62:6827/2104856662]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-03-24T11:12:54.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:54 smithi062 ceph-mon[29666]: osdmap e25: 4 total, 3 up, 4 in 2024-03-24T11:12:54.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:54 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T11:12:54.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:54 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T11:12:54.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:54 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:54.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:54 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:12:55.262 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:55 smithi139 ceph-mon[25719]: purged_snaps scrub starts 2024-03-24T11:12:55.262 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:55 smithi139 ceph-mon[25719]: purged_snaps scrub ok 2024-03-24T11:12:55.262 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:55 smithi139 ceph-mon[25719]: Detected new or changed devices on smithi062 2024-03-24T11:12:55.262 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:55 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T11:12:55.262 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:55 smithi139 ceph-mon[25719]: osd.3 [v2:172.21.15.62:6826/2104856662,v1:172.21.15.62:6827/2104856662] boot 2024-03-24T11:12:55.262 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:55 smithi139 ceph-mon[25719]: osdmap e26: 4 total, 4 up, 4 in 2024-03-24T11:12:55.262 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:55 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T11:12:55.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:55 smithi062 ceph-mon[24799]: purged_snaps scrub starts 2024-03-24T11:12:55.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:55 smithi062 ceph-mon[24799]: purged_snaps scrub ok 2024-03-24T11:12:55.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:55 smithi062 ceph-mon[24799]: Detected new or changed devices on smithi062 2024-03-24T11:12:55.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:55 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T11:12:55.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:55 smithi062 ceph-mon[24799]: osd.3 [v2:172.21.15.62:6826/2104856662,v1:172.21.15.62:6827/2104856662] boot 2024-03-24T11:12:55.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:55 smithi062 ceph-mon[24799]: osdmap e26: 4 total, 4 up, 4 in 2024-03-24T11:12:55.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:55 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T11:12:55.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:55 smithi062 ceph-mon[29666]: purged_snaps scrub starts 2024-03-24T11:12:55.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:55 smithi062 ceph-mon[29666]: purged_snaps scrub ok 2024-03-24T11:12:55.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:55 smithi062 ceph-mon[29666]: Detected new or changed devices on smithi062 2024-03-24T11:12:55.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:55 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T11:12:55.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:55 smithi062 ceph-mon[29666]: osd.3 [v2:172.21.15.62:6826/2104856662,v1:172.21.15.62:6827/2104856662] boot 2024-03-24T11:12:55.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:55 smithi062 ceph-mon[29666]: osdmap e26: 4 total, 4 up, 4 in 2024-03-24T11:12:55.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:55 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T11:12:56.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:56 smithi139 ceph-mon[25719]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2024-03-24T11:12:56.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:56 smithi139 ceph-mon[25719]: osdmap e27: 4 total, 4 up, 4 in 2024-03-24T11:12:56.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:56 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T11:12:56.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:56 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T11:12:56.316 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:56 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:12:56.411 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:56 smithi062 ceph-mon[24799]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2024-03-24T11:12:56.411 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:56 smithi062 ceph-mon[24799]: osdmap e27: 4 total, 4 up, 4 in 2024-03-24T11:12:56.411 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:56 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T11:12:56.411 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:56 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T11:12:56.411 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:56 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:12:56.411 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:56 smithi062 ceph-mon[29666]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2024-03-24T11:12:56.411 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:56 smithi062 ceph-mon[29666]: osdmap e27: 4 total, 4 up, 4 in 2024-03-24T11:12:56.411 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:56 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T11:12:56.412 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:56 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T11:12:56.412 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:56 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:12:57.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:57 smithi139 ceph-mon[25719]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:12:57.448 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:57 smithi062 ceph-mon[24799]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:12:57.448 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:57 smithi062 ceph-mon[29666]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:12:58.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:58 smithi062 ceph-mon[24799]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-03-24T11:12:58.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:58 smithi062 ceph-mon[24799]: from='client.? 172.21.15.139:0/2064765835' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c96358d-d11f-4f66-a433-3e4dc3024322"}]: dispatch 2024-03-24T11:12:58.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:58 smithi062 ceph-mon[24799]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c96358d-d11f-4f66-a433-3e4dc3024322"}]: dispatch 2024-03-24T11:12:58.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:58 smithi062 ceph-mon[24799]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8c96358d-d11f-4f66-a433-3e4dc3024322"}]': finished 2024-03-24T11:12:58.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:58 smithi062 ceph-mon[24799]: osdmap e28: 5 total, 4 up, 5 in 2024-03-24T11:12:58.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:58 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T11:12:58.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:58 smithi062 ceph-mon[29666]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-03-24T11:12:58.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:58 smithi062 ceph-mon[29666]: from='client.? 172.21.15.139:0/2064765835' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c96358d-d11f-4f66-a433-3e4dc3024322"}]: dispatch 2024-03-24T11:12:58.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:58 smithi062 ceph-mon[29666]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c96358d-d11f-4f66-a433-3e4dc3024322"}]: dispatch 2024-03-24T11:12:58.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:58 smithi062 ceph-mon[29666]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8c96358d-d11f-4f66-a433-3e4dc3024322"}]': finished 2024-03-24T11:12:58.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:58 smithi062 ceph-mon[29666]: osdmap e28: 5 total, 4 up, 5 in 2024-03-24T11:12:58.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:58 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T11:12:58.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:58 smithi139 ceph-mon[25719]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-03-24T11:12:58.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:58 smithi139 ceph-mon[25719]: from='client.? 172.21.15.139:0/2064765835' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c96358d-d11f-4f66-a433-3e4dc3024322"}]: dispatch 2024-03-24T11:12:58.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:58 smithi139 ceph-mon[25719]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c96358d-d11f-4f66-a433-3e4dc3024322"}]: dispatch 2024-03-24T11:12:58.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:58 smithi139 ceph-mon[25719]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8c96358d-d11f-4f66-a433-3e4dc3024322"}]': finished 2024-03-24T11:12:58.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:58 smithi139 ceph-mon[25719]: osdmap e28: 5 total, 4 up, 5 in 2024-03-24T11:12:58.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:58 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T11:12:59.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:12:59 smithi062 ceph-mon[24799]: from='client.? 172.21.15.139:0/365869922' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T11:12:59.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:12:59 smithi062 ceph-mon[29666]: from='client.? 172.21.15.139:0/365869922' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T11:12:59.607 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:12:59 smithi139 ceph-mon[25719]: from='client.? 172.21.15.139:0/365869922' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T11:13:00.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:00 smithi062 ceph-mon[24799]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-03-24T11:13:00.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:00 smithi062 ceph-mon[29666]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-03-24T11:13:00.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:00 smithi139 ceph-mon[25719]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-03-24T11:13:02.232 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:02 smithi139 ceph-mon[25719]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-03-24T11:13:02.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:02 smithi062 ceph-mon[24799]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-03-24T11:13:02.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:02 smithi062 ceph-mon[29666]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-03-24T11:13:03.976 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:13:03 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.139 - - [24/Mar/2024:11:13:03] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:13:04.432 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:04 smithi139 ceph-mon[25719]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-03-24T11:13:04.433 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:04 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:04.433 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:04 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:04.433 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:04 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-24T11:13:04.433 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:04 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:13:04.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:04 smithi062 ceph-mon[29666]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-03-24T11:13:04.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:04 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:04.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:04 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:04.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:04 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-24T11:13:04.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:04 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:13:04.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:04 smithi062 ceph-mon[24799]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-03-24T11:13:04.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:04 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:04.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:04 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:04.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:04 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-24T11:13:04.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:04 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:13:05.408 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:05 smithi139 ceph-mon[25719]: Deploying daemon osd.4 on smithi139 2024-03-24T11:13:05.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:05 smithi062 ceph-mon[24799]: Deploying daemon osd.4 on smithi139 2024-03-24T11:13:05.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:05 smithi062 ceph-mon[29666]: Deploying daemon osd.4 on smithi139 2024-03-24T11:13:06.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:06 smithi139 ceph-mon[25719]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-03-24T11:13:06.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:06 smithi062 ceph-mon[24799]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-03-24T11:13:06.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:06 smithi062 ceph-mon[29666]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-03-24T11:13:08.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:08 smithi062 ceph-mon[24799]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-03-24T11:13:08.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:08 smithi062 ceph-mon[29666]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-03-24T11:13:08.569 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:08 smithi139 ceph-mon[25719]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-03-24T11:13:10.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:10 smithi139 ceph-mon[25719]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-03-24T11:13:10.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:10 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:10.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:10 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:13:10.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:10 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:13:10.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:10 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:13:10.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:10 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:10.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:10 smithi062 ceph-mon[24799]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-03-24T11:13:10.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:10 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:10.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:10 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:13:10.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:10 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:13:10.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:10 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:13:10.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:10 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:10.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:10 smithi062 ceph-mon[29666]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-03-24T11:13:10.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:10 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:10.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:10 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:13:10.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:10 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:13:10.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:10 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:13:10.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:10 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:11.476 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:13:11 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.62 - - [24/Mar/2024:11:13:11] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:13:12.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:12 smithi062 ceph-mon[24799]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-03-24T11:13:12.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:12 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:12.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:12 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:12.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:12 smithi062 ceph-mon[29666]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-03-24T11:13:12.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:12 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:12.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:12 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:12.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:12 smithi139 ceph-mon[25719]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-03-24T11:13:12.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:12 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:12.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:12 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:13.241 INFO:teuthology.orchestra.run.smithi139.stdout:Created osd(s) 4 on host 'smithi139' 2024-03-24T11:13:13.803 DEBUG:teuthology.orchestra.run.smithi139:osd.4> sudo journalctl -f -n 0 -u ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.4.service 2024-03-24T11:13:13.806 INFO:tasks.cephadm:Deploying osd.5 on smithi139 with /dev/vg_nvme/lv_3... 2024-03-24T11:13:13.806 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-24T11:13:14.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:13 smithi139 ceph-mon[25719]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-03-24T11:13:14.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:13 smithi139 ceph-mon[25719]: from='osd.4 [v2:172.21.15.139:6800/3866478379,v1:172.21.15.139:6801/3866478379]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-24T11:13:14.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:13 smithi139 ceph-mon[25719]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-24T11:13:14.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:13 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:14.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:13 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:13:14.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:13 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:13:14.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:13 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:13:14.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:13 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:13:14.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:13 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:14.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:13 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:14.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:13 smithi062 ceph-mon[24799]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-03-24T11:13:14.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:13 smithi062 ceph-mon[24799]: from='osd.4 [v2:172.21.15.139:6800/3866478379,v1:172.21.15.139:6801/3866478379]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-24T11:13:14.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:13 smithi062 ceph-mon[24799]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-24T11:13:14.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:13 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:14.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:13 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:13:14.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:13 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:13:14.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:13 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:13:14.110 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:13 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:13:14.110 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:13 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:14.110 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:13 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:14.110 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:13 smithi062 ceph-mon[29666]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-03-24T11:13:14.110 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:13 smithi062 ceph-mon[29666]: from='osd.4 [v2:172.21.15.139:6800/3866478379,v1:172.21.15.139:6801/3866478379]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-24T11:13:14.110 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:13 smithi062 ceph-mon[29666]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-24T11:13:14.110 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:13 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:14.110 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:13 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:13:14.110 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:13 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:13:14.110 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:13 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:13:14.110 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:13 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:13:14.110 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:13 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:14.111 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:13 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:15.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:14 smithi139 ceph-mon[25719]: Adjusting osd_memory_target on smithi139 to 16064M 2024-03-24T11:13:15.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:14 smithi139 ceph-mon[25719]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-24T11:13:15.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:14 smithi139 ceph-mon[25719]: osdmap e29: 5 total, 4 up, 5 in 2024-03-24T11:13:15.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:14 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T11:13:15.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:14 smithi139 ceph-mon[25719]: from='osd.4 [v2:172.21.15.139:6800/3866478379,v1:172.21.15.139:6801/3866478379]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:13:15.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:14 smithi139 ceph-mon[25719]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:13:15.109 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:13:14 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4[30328]: 2024-03-24T11:13:14.832+0000 7fdc39581700 -1 osd.4 0 waiting for initial osdmap 2024-03-24T11:13:15.109 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:13:14 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4[30328]: 2024-03-24T11:13:14.836+0000 7fdc35f1c700 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T11:13:15.130 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:14 smithi062 ceph-mon[29666]: Adjusting osd_memory_target on smithi139 to 16064M 2024-03-24T11:13:15.131 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:14 smithi062 ceph-mon[29666]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-24T11:13:15.131 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:14 smithi062 ceph-mon[29666]: osdmap e29: 5 total, 4 up, 5 in 2024-03-24T11:13:15.131 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:14 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T11:13:15.131 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:14 smithi062 ceph-mon[29666]: from='osd.4 [v2:172.21.15.139:6800/3866478379,v1:172.21.15.139:6801/3866478379]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:13:15.131 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:14 smithi062 ceph-mon[29666]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:13:15.131 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:14 smithi062 ceph-mon[24799]: Adjusting osd_memory_target on smithi139 to 16064M 2024-03-24T11:13:15.131 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:14 smithi062 ceph-mon[24799]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-24T11:13:15.131 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:14 smithi062 ceph-mon[24799]: osdmap e29: 5 total, 4 up, 5 in 2024-03-24T11:13:15.131 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:14 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T11:13:15.131 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:14 smithi062 ceph-mon[24799]: from='osd.4 [v2:172.21.15.139:6800/3866478379,v1:172.21.15.139:6801/3866478379]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:13:15.132 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:14 smithi062 ceph-mon[24799]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:13:15.740 INFO:teuthology.orchestra.run.smithi139.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-24T11:13:15.740 INFO:teuthology.orchestra.run.smithi139.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-24T11:13:15.740 INFO:teuthology.orchestra.run.smithi139.stderr: stderr: 10+0 records in 2024-03-24T11:13:15.740 INFO:teuthology.orchestra.run.smithi139.stderr:10+0 records out 2024-03-24T11:13:15.741 INFO:teuthology.orchestra.run.smithi139.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.013957 s, 751 MB/s 2024-03-24T11:13:15.741 INFO:teuthology.orchestra.run.smithi139.stderr:--> Zapping successful for: 2024-03-24T11:13:16.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:15 smithi139 ceph-mon[25719]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-03-24T11:13:16.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:15 smithi139 ceph-mon[25719]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-03-24T11:13:16.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:15 smithi139 ceph-mon[25719]: osdmap e30: 5 total, 4 up, 5 in 2024-03-24T11:13:16.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:15 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T11:13:16.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:15 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T11:13:16.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:15 smithi139 ceph-mon[25719]: osd.4 [v2:172.21.15.139:6800/3866478379,v1:172.21.15.139:6801/3866478379] boot 2024-03-24T11:13:16.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:15 smithi139 ceph-mon[25719]: osdmap e31: 5 total, 5 up, 5 in 2024-03-24T11:13:16.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:15 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T11:13:16.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:15 smithi062 ceph-mon[29666]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-03-24T11:13:16.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:15 smithi062 ceph-mon[29666]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-03-24T11:13:16.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:15 smithi062 ceph-mon[29666]: osdmap e30: 5 total, 4 up, 5 in 2024-03-24T11:13:16.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:15 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T11:13:16.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:15 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T11:13:16.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:15 smithi062 ceph-mon[29666]: osd.4 [v2:172.21.15.139:6800/3866478379,v1:172.21.15.139:6801/3866478379] boot 2024-03-24T11:13:16.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:15 smithi062 ceph-mon[29666]: osdmap e31: 5 total, 5 up, 5 in 2024-03-24T11:13:16.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:15 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T11:13:16.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:15 smithi062 ceph-mon[24799]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2024-03-24T11:13:16.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:15 smithi062 ceph-mon[24799]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-03-24T11:13:16.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:15 smithi062 ceph-mon[24799]: osdmap e30: 5 total, 4 up, 5 in 2024-03-24T11:13:16.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:15 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T11:13:16.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:15 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T11:13:16.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:15 smithi062 ceph-mon[24799]: osd.4 [v2:172.21.15.139:6800/3866478379,v1:172.21.15.139:6801/3866478379] boot 2024-03-24T11:13:16.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:15 smithi062 ceph-mon[24799]: osdmap e31: 5 total, 5 up, 5 in 2024-03-24T11:13:16.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:15 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T11:13:16.340 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi139:vg_nvme/lv_3 2024-03-24T11:13:16.936 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:16 smithi139 ceph-mon[25719]: purged_snaps scrub starts 2024-03-24T11:13:16.936 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:16 smithi139 ceph-mon[25719]: purged_snaps scrub ok 2024-03-24T11:13:16.936 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:16 smithi139 ceph-mon[25719]: osdmap e32: 5 total, 5 up, 5 in 2024-03-24T11:13:17.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:16 smithi062 ceph-mon[29666]: purged_snaps scrub starts 2024-03-24T11:13:17.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:16 smithi062 ceph-mon[29666]: purged_snaps scrub ok 2024-03-24T11:13:17.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:16 smithi062 ceph-mon[29666]: osdmap e32: 5 total, 5 up, 5 in 2024-03-24T11:13:17.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:16 smithi062 ceph-mon[24799]: purged_snaps scrub starts 2024-03-24T11:13:17.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:16 smithi062 ceph-mon[24799]: purged_snaps scrub ok 2024-03-24T11:13:17.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:16 smithi062 ceph-mon[24799]: osdmap e32: 5 total, 5 up, 5 in 2024-03-24T11:13:18.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:17 smithi139 ceph-mon[25719]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2024-03-24T11:13:18.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:17 smithi139 ceph-mon[25719]: osdmap e33: 5 total, 5 up, 5 in 2024-03-24T11:13:18.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:17 smithi062 ceph-mon[29666]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2024-03-24T11:13:18.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:17 smithi062 ceph-mon[29666]: osdmap e33: 5 total, 5 up, 5 in 2024-03-24T11:13:18.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:17 smithi062 ceph-mon[24799]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2024-03-24T11:13:18.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:17 smithi062 ceph-mon[24799]: osdmap e33: 5 total, 5 up, 5 in 2024-03-24T11:13:19.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:18 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T11:13:19.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:18 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T11:13:19.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:18 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:13:19.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:18 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T11:13:19.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:18 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T11:13:19.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:18 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:13:19.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:18 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T11:13:19.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:18 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T11:13:19.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:18 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:13:19.885 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:19 smithi139 ceph-mon[25719]: from='client.24245 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:13:19.885 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:19 smithi139 ceph-mon[25719]: pgmap v113: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-03-24T11:13:19.885 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:19 smithi139 ceph-mon[25719]: osdmap e34: 5 total, 5 up, 5 in 2024-03-24T11:13:20.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:19 smithi062 ceph-mon[24799]: from='client.24245 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:13:20.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:19 smithi062 ceph-mon[24799]: pgmap v113: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-03-24T11:13:20.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:19 smithi062 ceph-mon[24799]: osdmap e34: 5 total, 5 up, 5 in 2024-03-24T11:13:20.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:19 smithi062 ceph-mon[29666]: from='client.24245 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:13:20.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:19 smithi062 ceph-mon[29666]: pgmap v113: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-03-24T11:13:20.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:19 smithi062 ceph-mon[29666]: osdmap e34: 5 total, 5 up, 5 in 2024-03-24T11:13:21.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:20 smithi139 ceph-mon[25719]: osdmap e35: 5 total, 5 up, 5 in 2024-03-24T11:13:21.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:20 smithi139 ceph-mon[25719]: from='client.? 172.21.15.139:0/1706661739' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "adf8b0ea-4d03-4bc4-bca4-2b6aad9e967d"}]: dispatch 2024-03-24T11:13:21.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:20 smithi139 ceph-mon[25719]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "adf8b0ea-4d03-4bc4-bca4-2b6aad9e967d"}]: dispatch 2024-03-24T11:13:21.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:20 smithi139 ceph-mon[25719]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "adf8b0ea-4d03-4bc4-bca4-2b6aad9e967d"}]': finished 2024-03-24T11:13:21.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:20 smithi139 ceph-mon[25719]: osdmap e36: 6 total, 5 up, 6 in 2024-03-24T11:13:21.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:20 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T11:13:21.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:20 smithi062 ceph-mon[24799]: osdmap e35: 5 total, 5 up, 5 in 2024-03-24T11:13:21.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:20 smithi062 ceph-mon[24799]: from='client.? 172.21.15.139:0/1706661739' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "adf8b0ea-4d03-4bc4-bca4-2b6aad9e967d"}]: dispatch 2024-03-24T11:13:21.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:20 smithi062 ceph-mon[24799]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "adf8b0ea-4d03-4bc4-bca4-2b6aad9e967d"}]: dispatch 2024-03-24T11:13:21.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:20 smithi062 ceph-mon[24799]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "adf8b0ea-4d03-4bc4-bca4-2b6aad9e967d"}]': finished 2024-03-24T11:13:21.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:20 smithi062 ceph-mon[24799]: osdmap e36: 6 total, 5 up, 6 in 2024-03-24T11:13:21.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:20 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T11:13:21.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:20 smithi062 ceph-mon[29666]: osdmap e35: 5 total, 5 up, 5 in 2024-03-24T11:13:21.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:20 smithi062 ceph-mon[29666]: from='client.? 172.21.15.139:0/1706661739' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "adf8b0ea-4d03-4bc4-bca4-2b6aad9e967d"}]: dispatch 2024-03-24T11:13:21.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:20 smithi062 ceph-mon[29666]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "adf8b0ea-4d03-4bc4-bca4-2b6aad9e967d"}]: dispatch 2024-03-24T11:13:21.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:20 smithi062 ceph-mon[29666]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "adf8b0ea-4d03-4bc4-bca4-2b6aad9e967d"}]': finished 2024-03-24T11:13:21.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:20 smithi062 ceph-mon[29666]: osdmap e36: 6 total, 5 up, 6 in 2024-03-24T11:13:21.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:20 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T11:13:22.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:21 smithi139 ceph-mon[25719]: pgmap v116: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-03-24T11:13:22.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:21 smithi139 ceph-mon[25719]: from='client.? 172.21.15.139:0/695655002' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T11:13:22.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:21 smithi062 ceph-mon[24799]: pgmap v116: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-03-24T11:13:22.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:21 smithi062 ceph-mon[24799]: from='client.? 172.21.15.139:0/695655002' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T11:13:22.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:21 smithi062 ceph-mon[29666]: pgmap v116: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-03-24T11:13:22.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:21 smithi062 ceph-mon[29666]: from='client.? 172.21.15.139:0/695655002' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T11:13:24.159 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:23 smithi139 ceph-mon[25719]: pgmap v118: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-03-24T11:13:24.159 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:23 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:24.159 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:23 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:24.159 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:23 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:13:24.159 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:23 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:13:24.159 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:23 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:13:24.159 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:23 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:24.159 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:23 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-24T11:13:24.160 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:23 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-24T11:13:24.226 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:13:23 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.139 - - [24/Mar/2024:11:13:23] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:13:24.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:23 smithi062 ceph-mon[29666]: pgmap v118: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-03-24T11:13:24.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:23 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:24.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:23 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:24.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:23 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:13:24.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:23 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:13:24.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:23 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:13:24.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:23 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:24.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:23 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-24T11:13:24.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:23 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-24T11:13:24.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:23 smithi062 ceph-mon[24799]: pgmap v118: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-03-24T11:13:24.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:23 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:24.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:23 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:24.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:23 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:13:24.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:23 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:13:24.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:23 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:13:24.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:23 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:24.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:23 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-24T11:13:24.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:23 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-24T11:13:24.727 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:13:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.139 - - [24/Mar/2024:11:13:24] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:13:25.226 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:24 smithi062 ceph-mon[29666]: Detected new or changed devices on smithi139 2024-03-24T11:13:25.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:24 smithi062 ceph-mon[29666]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2024-03-24T11:13:25.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:24 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:25.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:24 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:25.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:24 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:13:25.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:24 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:13:25.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:24 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:13:25.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:24 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:25.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:24 smithi062 ceph-mon[24799]: Detected new or changed devices on smithi139 2024-03-24T11:13:25.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:24 smithi062 ceph-mon[24799]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2024-03-24T11:13:25.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:24 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:25.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:24 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:25.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:24 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:13:25.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:24 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:13:25.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:24 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:13:25.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:24 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:25.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:24 smithi139 ceph-mon[25719]: Detected new or changed devices on smithi139 2024-03-24T11:13:25.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:24 smithi139 ceph-mon[25719]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2024-03-24T11:13:25.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:24 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:25.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:24 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:25.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:24 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:13:25.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:24 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:13:25.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:24 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:13:25.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:24 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:26.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:25 smithi139 ceph-mon[25719]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2024-03-24T11:13:26.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:25 smithi062 ceph-mon[24799]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2024-03-24T11:13:26.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:25 smithi062 ceph-mon[29666]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2024-03-24T11:13:28.018 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:27 smithi139 ceph-mon[25719]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2024-03-24T11:13:28.018 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:27 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-24T11:13:28.018 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:27 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:13:28.019 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:27 smithi139 ceph-mon[25719]: Deploying daemon osd.5 on smithi139 2024-03-24T11:13:28.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:27 smithi062 ceph-mon[24799]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2024-03-24T11:13:28.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:27 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-24T11:13:28.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:27 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:13:28.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:27 smithi062 ceph-mon[24799]: Deploying daemon osd.5 on smithi139 2024-03-24T11:13:28.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:27 smithi062 ceph-mon[29666]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2024-03-24T11:13:28.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:27 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-24T11:13:28.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:27 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:13:28.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:27 smithi062 ceph-mon[29666]: Deploying daemon osd.5 on smithi139 2024-03-24T11:13:30.022 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:29 smithi139 ceph-mon[25719]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2024-03-24T11:13:30.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:29 smithi062 ceph-mon[24799]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2024-03-24T11:13:30.215 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:29 smithi062 ceph-mon[29666]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2024-03-24T11:13:31.726 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:13:31 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.62 - - [24/Mar/2024:11:13:31] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:13:31.993 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:31 smithi139 ceph-mon[25719]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2024-03-24T11:13:31.993 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:31 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:31.993 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:31 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:32.092 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:31 smithi062 ceph-mon[24799]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2024-03-24T11:13:32.092 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:31 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:32.092 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:31 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:32.093 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:31 smithi062 ceph-mon[29666]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2024-03-24T11:13:32.093 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:31 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:32.093 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:31 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:34.067 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:33 smithi139 ceph-mon[25719]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2024-03-24T11:13:34.067 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:33 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:34.067 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:33 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:13:34.067 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:33 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:13:34.067 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:33 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:13:34.067 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:33 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:34.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:33 smithi062 ceph-mon[24799]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2024-03-24T11:13:34.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:33 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:34.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:33 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:13:34.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:33 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:13:34.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:33 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:13:34.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:33 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:34.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:33 smithi062 ceph-mon[29666]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2024-03-24T11:13:34.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:33 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:34.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:33 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:13:34.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:33 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:13:34.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:33 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:13:34.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:33 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:36.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:35 smithi062 ceph-mon[24799]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2024-03-24T11:13:36.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:35 smithi062 ceph-mon[29666]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2024-03-24T11:13:36.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:35 smithi139 ceph-mon[25719]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2024-03-24T11:13:36.738 INFO:teuthology.orchestra.run.smithi139.stdout:Created osd(s) 5 on host 'smithi139' 2024-03-24T11:13:37.091 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:36 smithi139 ceph-mon[25719]: from='osd.5 [v2:172.21.15.139:6808/1384318027,v1:172.21.15.139:6809/1384318027]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-24T11:13:37.091 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:36 smithi139 ceph-mon[25719]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-24T11:13:37.091 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:36 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:37.091 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:36 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:13:37.091 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:36 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:13:37.091 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:36 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:13:37.091 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:36 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:13:37.091 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:36 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:13:37.091 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:36 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:37.091 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:36 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:37.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:36 smithi062 ceph-mon[24799]: from='osd.5 [v2:172.21.15.139:6808/1384318027,v1:172.21.15.139:6809/1384318027]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-24T11:13:37.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:36 smithi062 ceph-mon[24799]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-24T11:13:37.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:36 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:37.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:36 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:13:37.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:36 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:13:37.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:36 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:13:37.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:36 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:13:37.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:36 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:13:37.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:36 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:37.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:36 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:37.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:36 smithi062 ceph-mon[29666]: from='osd.5 [v2:172.21.15.139:6808/1384318027,v1:172.21.15.139:6809/1384318027]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-24T11:13:37.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:36 smithi062 ceph-mon[29666]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-24T11:13:37.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:36 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:37.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:36 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:13:37.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:36 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:13:37.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:36 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:13:37.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:36 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:13:37.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:36 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:13:37.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:36 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:37.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:36 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:37.385 DEBUG:teuthology.orchestra.run.smithi139:osd.5> sudo journalctl -f -n 0 -u ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.5.service 2024-03-24T11:13:37.388 INFO:tasks.cephadm:Deploying osd.6 on smithi139 with /dev/vg_nvme/lv_2... 2024-03-24T11:13:37.388 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-24T11:13:38.027 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:37 smithi139 ceph-mon[25719]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2024-03-24T11:13:38.027 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:37 smithi139 ceph-mon[25719]: Adjusting osd_memory_target on smithi139 to 8032M 2024-03-24T11:13:38.027 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:37 smithi139 ceph-mon[25719]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-24T11:13:38.027 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:37 smithi139 ceph-mon[25719]: osdmap e37: 6 total, 5 up, 6 in 2024-03-24T11:13:38.027 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:37 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T11:13:38.027 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:37 smithi139 ceph-mon[25719]: from='osd.5 [v2:172.21.15.139:6808/1384318027,v1:172.21.15.139:6809/1384318027]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:13:38.028 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:37 smithi139 ceph-mon[25719]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:13:38.028 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:13:37 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5[33215]: 2024-03-24T11:13:37.925+0000 7f7edb868700 -1 osd.5 0 waiting for initial osdmap 2024-03-24T11:13:38.028 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:13:37 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5[33215]: 2024-03-24T11:13:37.931+0000 7f7ed7a02700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T11:13:38.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:37 smithi062 ceph-mon[24799]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2024-03-24T11:13:38.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:37 smithi062 ceph-mon[24799]: Adjusting osd_memory_target on smithi139 to 8032M 2024-03-24T11:13:38.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:37 smithi062 ceph-mon[24799]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-24T11:13:38.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:37 smithi062 ceph-mon[24799]: osdmap e37: 6 total, 5 up, 6 in 2024-03-24T11:13:38.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:37 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T11:13:38.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:37 smithi062 ceph-mon[24799]: from='osd.5 [v2:172.21.15.139:6808/1384318027,v1:172.21.15.139:6809/1384318027]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:13:38.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:37 smithi062 ceph-mon[24799]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:13:38.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:37 smithi062 ceph-mon[29666]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2024-03-24T11:13:38.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:37 smithi062 ceph-mon[29666]: Adjusting osd_memory_target on smithi139 to 8032M 2024-03-24T11:13:38.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:37 smithi062 ceph-mon[29666]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-24T11:13:38.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:37 smithi062 ceph-mon[29666]: osdmap e37: 6 total, 5 up, 6 in 2024-03-24T11:13:38.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:37 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T11:13:38.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:37 smithi062 ceph-mon[29666]: from='osd.5 [v2:172.21.15.139:6808/1384318027,v1:172.21.15.139:6809/1384318027]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:13:38.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:37 smithi062 ceph-mon[29666]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:13:39.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:38 smithi062 ceph-mon[24799]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-03-24T11:13:39.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:38 smithi062 ceph-mon[24799]: osdmap e38: 6 total, 5 up, 6 in 2024-03-24T11:13:39.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:38 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T11:13:39.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:38 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T11:13:39.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:38 smithi062 ceph-mon[29666]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-03-24T11:13:39.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:38 smithi062 ceph-mon[29666]: osdmap e38: 6 total, 5 up, 6 in 2024-03-24T11:13:39.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:38 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T11:13:39.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:38 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T11:13:39.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:38 smithi139 ceph-mon[25719]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-03-24T11:13:39.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:38 smithi139 ceph-mon[25719]: osdmap e38: 6 total, 5 up, 6 in 2024-03-24T11:13:39.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:38 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T11:13:39.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:38 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T11:13:39.482 INFO:teuthology.orchestra.run.smithi139.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-24T11:13:39.482 INFO:teuthology.orchestra.run.smithi139.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-24T11:13:39.482 INFO:teuthology.orchestra.run.smithi139.stderr: stderr: 10+0 records in 2024-03-24T11:13:39.482 INFO:teuthology.orchestra.run.smithi139.stderr:10+0 records out 2024-03-24T11:13:39.482 INFO:teuthology.orchestra.run.smithi139.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0123016 s, 852 MB/s 2024-03-24T11:13:39.482 INFO:teuthology.orchestra.run.smithi139.stderr:--> Zapping successful for: 2024-03-24T11:13:40.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:39 smithi139 ceph-mon[25719]: purged_snaps scrub starts 2024-03-24T11:13:40.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:39 smithi139 ceph-mon[25719]: purged_snaps scrub ok 2024-03-24T11:13:40.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:39 smithi139 ceph-mon[25719]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2024-03-24T11:13:40.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:39 smithi139 ceph-mon[25719]: osd.5 [v2:172.21.15.139:6808/1384318027,v1:172.21.15.139:6809/1384318027] boot 2024-03-24T11:13:40.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:39 smithi139 ceph-mon[25719]: osdmap e39: 6 total, 6 up, 6 in 2024-03-24T11:13:40.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:39 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T11:13:40.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:39 smithi139 ceph-mon[25719]: osdmap e40: 6 total, 6 up, 6 in 2024-03-24T11:13:40.146 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi139:vg_nvme/lv_2 2024-03-24T11:13:40.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:39 smithi062 ceph-mon[24799]: purged_snaps scrub starts 2024-03-24T11:13:40.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:39 smithi062 ceph-mon[24799]: purged_snaps scrub ok 2024-03-24T11:13:40.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:39 smithi062 ceph-mon[24799]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2024-03-24T11:13:40.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:39 smithi062 ceph-mon[24799]: osd.5 [v2:172.21.15.139:6808/1384318027,v1:172.21.15.139:6809/1384318027] boot 2024-03-24T11:13:40.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:39 smithi062 ceph-mon[24799]: osdmap e39: 6 total, 6 up, 6 in 2024-03-24T11:13:40.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:39 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T11:13:40.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:39 smithi062 ceph-mon[24799]: osdmap e40: 6 total, 6 up, 6 in 2024-03-24T11:13:40.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:39 smithi062 ceph-mon[29666]: purged_snaps scrub starts 2024-03-24T11:13:40.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:39 smithi062 ceph-mon[29666]: purged_snaps scrub ok 2024-03-24T11:13:40.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:39 smithi062 ceph-mon[29666]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2024-03-24T11:13:40.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:39 smithi062 ceph-mon[29666]: osd.5 [v2:172.21.15.139:6808/1384318027,v1:172.21.15.139:6809/1384318027] boot 2024-03-24T11:13:40.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:39 smithi062 ceph-mon[29666]: osdmap e39: 6 total, 6 up, 6 in 2024-03-24T11:13:40.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:39 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T11:13:40.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:39 smithi062 ceph-mon[29666]: osdmap e40: 6 total, 6 up, 6 in 2024-03-24T11:13:42.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:41 smithi062 ceph-mon[24799]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2024-03-24T11:13:42.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:41 smithi062 ceph-mon[24799]: osdmap e41: 6 total, 6 up, 6 in 2024-03-24T11:13:42.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:41 smithi062 ceph-mon[29666]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2024-03-24T11:13:42.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:41 smithi062 ceph-mon[29666]: osdmap e41: 6 total, 6 up, 6 in 2024-03-24T11:13:42.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:41 smithi139 ceph-mon[25719]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2024-03-24T11:13:42.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:41 smithi139 ceph-mon[25719]: osdmap e41: 6 total, 6 up, 6 in 2024-03-24T11:13:43.051 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:42 smithi139 ceph-mon[25719]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:13:43.051 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:42 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T11:13:43.052 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:42 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T11:13:43.052 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:42 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:13:43.052 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:42 smithi139 ceph-mon[25719]: osdmap e42: 6 total, 6 up, 6 in 2024-03-24T11:13:43.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:42 smithi062 ceph-mon[24799]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:13:43.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:42 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T11:13:43.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:42 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T11:13:43.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:42 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:13:43.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:42 smithi062 ceph-mon[24799]: osdmap e42: 6 total, 6 up, 6 in 2024-03-24T11:13:43.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:42 smithi062 ceph-mon[29666]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:13:43.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:42 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T11:13:43.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:42 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T11:13:43.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:42 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:13:43.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:42 smithi062 ceph-mon[29666]: osdmap e42: 6 total, 6 up, 6 in 2024-03-24T11:13:44.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:43 smithi139 ceph-mon[25719]: pgmap v134: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2024-03-24T11:13:44.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:43 smithi139 ceph-mon[25719]: osdmap e43: 6 total, 6 up, 6 in 2024-03-24T11:13:44.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:43 smithi062 ceph-mon[24799]: pgmap v134: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2024-03-24T11:13:44.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:43 smithi062 ceph-mon[24799]: osdmap e43: 6 total, 6 up, 6 in 2024-03-24T11:13:44.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:43 smithi062 ceph-mon[29666]: pgmap v134: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2024-03-24T11:13:44.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:43 smithi062 ceph-mon[29666]: osdmap e43: 6 total, 6 up, 6 in 2024-03-24T11:13:44.762 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:13:44 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.139 - - [24/Mar/2024:11:13:44] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:13:45.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:44 smithi062 ceph-mon[24799]: from='client.? 172.21.15.139:0/1367559943' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f9790a56-fa0c-4908-8fbc-c347ecefa2df"}]: dispatch 2024-03-24T11:13:45.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:44 smithi062 ceph-mon[24799]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f9790a56-fa0c-4908-8fbc-c347ecefa2df"}]: dispatch 2024-03-24T11:13:45.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:44 smithi062 ceph-mon[24799]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f9790a56-fa0c-4908-8fbc-c347ecefa2df"}]': finished 2024-03-24T11:13:45.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:44 smithi062 ceph-mon[24799]: osdmap e44: 7 total, 6 up, 7 in 2024-03-24T11:13:45.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:44 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T11:13:45.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:44 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:45.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:44 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:45.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:44 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:13:45.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:44 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:13:45.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:44 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:13:45.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:44 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:45.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:44 smithi062 ceph-mon[29666]: from='client.? 172.21.15.139:0/1367559943' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f9790a56-fa0c-4908-8fbc-c347ecefa2df"}]: dispatch 2024-03-24T11:13:45.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:44 smithi062 ceph-mon[29666]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f9790a56-fa0c-4908-8fbc-c347ecefa2df"}]: dispatch 2024-03-24T11:13:45.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:44 smithi062 ceph-mon[29666]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f9790a56-fa0c-4908-8fbc-c347ecefa2df"}]': finished 2024-03-24T11:13:45.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:44 smithi062 ceph-mon[29666]: osdmap e44: 7 total, 6 up, 7 in 2024-03-24T11:13:45.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:44 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T11:13:45.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:44 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:45.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:44 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:45.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:44 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:13:45.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:44 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:13:45.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:44 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:13:45.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:44 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:45.234 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:44 smithi139 ceph-mon[25719]: from='client.? 172.21.15.139:0/1367559943' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f9790a56-fa0c-4908-8fbc-c347ecefa2df"}]: dispatch 2024-03-24T11:13:45.234 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:44 smithi139 ceph-mon[25719]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f9790a56-fa0c-4908-8fbc-c347ecefa2df"}]: dispatch 2024-03-24T11:13:45.234 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:44 smithi139 ceph-mon[25719]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f9790a56-fa0c-4908-8fbc-c347ecefa2df"}]': finished 2024-03-24T11:13:45.234 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:44 smithi139 ceph-mon[25719]: osdmap e44: 7 total, 6 up, 7 in 2024-03-24T11:13:45.234 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:44 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T11:13:45.234 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:44 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:45.234 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:44 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:45.234 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:44 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:13:45.234 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:44 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:13:45.234 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:44 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:13:45.235 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:44 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:45.942 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:13:45 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.139 - - [24/Mar/2024:11:13:45] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:13:46.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:45 smithi062 ceph-mon[29666]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-03-24T11:13:46.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:45 smithi062 ceph-mon[29666]: Detected new or changed devices on smithi139 2024-03-24T11:13:46.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:45 smithi062 ceph-mon[29666]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-03-24T11:13:46.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:45 smithi062 ceph-mon[29666]: from='client.? 172.21.15.139:0/2416023460' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T11:13:46.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:45 smithi062 ceph-mon[29666]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-03-24T11:13:46.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:45 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:46.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:45 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:46.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:45 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:13:46.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:45 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:13:46.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:45 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:13:46.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:45 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:46.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:45 smithi062 ceph-mon[24799]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-03-24T11:13:46.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:45 smithi062 ceph-mon[24799]: Detected new or changed devices on smithi139 2024-03-24T11:13:46.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:45 smithi062 ceph-mon[24799]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-03-24T11:13:46.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:45 smithi062 ceph-mon[24799]: from='client.? 172.21.15.139:0/2416023460' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T11:13:46.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:45 smithi062 ceph-mon[24799]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-03-24T11:13:46.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:45 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:46.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:45 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:46.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:45 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:13:46.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:45 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:13:46.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:45 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:13:46.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:45 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:46.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:45 smithi139 ceph-mon[25719]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-03-24T11:13:46.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:45 smithi139 ceph-mon[25719]: Detected new or changed devices on smithi139 2024-03-24T11:13:46.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:45 smithi139 ceph-mon[25719]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-03-24T11:13:46.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:45 smithi139 ceph-mon[25719]: from='client.? 172.21.15.139:0/2416023460' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T11:13:46.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:45 smithi139 ceph-mon[25719]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-03-24T11:13:46.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:45 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:46.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:45 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:46.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:45 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:13:46.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:45 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:13:46.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:45 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:13:46.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:45 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:47.202 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:46 smithi139 ceph-mon[25719]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-03-24T11:13:47.202 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:46 smithi139 ceph-mon[25719]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-03-24T11:13:47.202 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:46 smithi139 ceph-mon[25719]: Cluster is now healthy 2024-03-24T11:13:47.226 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:46 smithi062 ceph-mon[29666]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-03-24T11:13:47.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:46 smithi062 ceph-mon[29666]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-03-24T11:13:47.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:46 smithi062 ceph-mon[29666]: Cluster is now healthy 2024-03-24T11:13:47.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:46 smithi062 ceph-mon[24799]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-03-24T11:13:47.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:46 smithi062 ceph-mon[24799]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-03-24T11:13:47.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:46 smithi062 ceph-mon[24799]: Cluster is now healthy 2024-03-24T11:13:49.019 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:48 smithi139 ceph-mon[25719]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-03-24T11:13:49.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:48 smithi062 ceph-mon[24799]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-03-24T11:13:49.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:48 smithi062 ceph-mon[29666]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-03-24T11:13:51.002 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:50 smithi139 ceph-mon[25719]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-03-24T11:13:51.002 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:50 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-24T11:13:51.002 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:50 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:13:51.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:50 smithi062 ceph-mon[24799]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-03-24T11:13:51.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:50 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-24T11:13:51.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:50 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:13:51.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:50 smithi062 ceph-mon[29666]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-03-24T11:13:51.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:50 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-24T11:13:51.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:50 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:13:51.726 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:13:51 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.62 - - [24/Mar/2024:11:13:51] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:13:52.176 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:51 smithi139 ceph-mon[25719]: Deploying daemon osd.6 on smithi139 2024-03-24T11:13:52.176 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:51 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:52.176 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:51 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:52.223 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:51 smithi062 ceph-mon[24799]: Deploying daemon osd.6 on smithi139 2024-03-24T11:13:52.224 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:51 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:52.224 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:51 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:52.224 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:51 smithi062 ceph-mon[29666]: Deploying daemon osd.6 on smithi139 2024-03-24T11:13:52.224 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:51 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:52.224 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:51 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:53.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:52 smithi062 ceph-mon[24799]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-03-24T11:13:53.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:52 smithi062 ceph-mon[29666]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-03-24T11:13:53.232 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:52 smithi139 ceph-mon[25719]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-03-24T11:13:55.056 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:54 smithi139 ceph-mon[25719]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-03-24T11:13:55.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:54 smithi062 ceph-mon[24799]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-03-24T11:13:55.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:54 smithi062 ceph-mon[29666]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-03-24T11:13:57.062 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:56 smithi139 ceph-mon[25719]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-03-24T11:13:57.062 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:56 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:57.062 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:56 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:13:57.062 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:56 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:13:57.062 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:56 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:13:57.062 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:56 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:57.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:56 smithi062 ceph-mon[24799]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-03-24T11:13:57.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:56 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:57.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:56 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:13:57.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:56 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:13:57.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:56 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:13:57.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:56 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:57.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:56 smithi062 ceph-mon[29666]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-03-24T11:13:57.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:56 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:57.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:56 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:13:57.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:56 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:13:57.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:56 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:13:57.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:56 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:13:59.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:58 smithi139 ceph-mon[25719]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-03-24T11:13:59.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:58 smithi062 ceph-mon[24799]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-03-24T11:13:59.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:58 smithi062 ceph-mon[29666]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-03-24T11:14:00.226 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:59 smithi139 ceph-mon[25719]: from='osd.6 [v2:172.21.15.139:6816/2768426846,v1:172.21.15.139:6817/2768426846]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-24T11:14:00.226 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:13:59 smithi139 ceph-mon[25719]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-24T11:14:00.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:59 smithi062 ceph-mon[24799]: from='osd.6 [v2:172.21.15.139:6816/2768426846,v1:172.21.15.139:6817/2768426846]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-24T11:14:00.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:13:59 smithi062 ceph-mon[24799]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-24T11:14:00.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:59 smithi062 ceph-mon[29666]: from='osd.6 [v2:172.21.15.139:6816/2768426846,v1:172.21.15.139:6817/2768426846]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-24T11:14:00.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:13:59 smithi062 ceph-mon[29666]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-24T11:14:00.258 INFO:teuthology.orchestra.run.smithi139.stdout:Created osd(s) 6 on host 'smithi139' 2024-03-24T11:14:00.877 DEBUG:teuthology.orchestra.run.smithi139:osd.6> sudo journalctl -f -n 0 -u ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.6.service 2024-03-24T11:14:00.881 INFO:tasks.cephadm:Deploying osd.7 on smithi139 with /dev/vg_nvme/lv_1... 2024-03-24T11:14:00.881 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-24T11:14:01.108 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:14:00 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[36139]: 2024-03-24T11:14:00.976+0000 7f4ab53df700 -1 osd.6 0 waiting for initial osdmap 2024-03-24T11:14:01.108 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:14:00 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[36139]: 2024-03-24T11:14:00.982+0000 7f4ab0577700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T11:14:01.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:00 smithi139 ceph-mon[25719]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-03-24T11:14:01.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:00 smithi139 ceph-mon[25719]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-24T11:14:01.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:00 smithi139 ceph-mon[25719]: osdmap e45: 7 total, 6 up, 7 in 2024-03-24T11:14:01.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:00 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T11:14:01.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:00 smithi139 ceph-mon[25719]: from='osd.6 [v2:172.21.15.139:6816/2768426846,v1:172.21.15.139:6817/2768426846]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:14:01.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:00 smithi139 ceph-mon[25719]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:14:01.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:00 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:01.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:00 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:14:01.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:00 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:14:01.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:00 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:14:01.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:00 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:14:01.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:00 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:14:01.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:00 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:14:01.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:00 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:01.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:00 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:01.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:00 smithi062 ceph-mon[24799]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-03-24T11:14:01.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:00 smithi062 ceph-mon[24799]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-24T11:14:01.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:00 smithi062 ceph-mon[24799]: osdmap e45: 7 total, 6 up, 7 in 2024-03-24T11:14:01.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:00 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T11:14:01.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:00 smithi062 ceph-mon[24799]: from='osd.6 [v2:172.21.15.139:6816/2768426846,v1:172.21.15.139:6817/2768426846]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:14:01.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:00 smithi062 ceph-mon[24799]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:14:01.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:00 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:01.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:00 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:14:01.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:00 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:14:01.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:00 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:14:01.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:00 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:14:01.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:00 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:14:01.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:00 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:14:01.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:00 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:01.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:00 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:01.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:00 smithi062 ceph-mon[29666]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-03-24T11:14:01.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:00 smithi062 ceph-mon[29666]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-24T11:14:01.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:00 smithi062 ceph-mon[29666]: osdmap e45: 7 total, 6 up, 7 in 2024-03-24T11:14:01.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:00 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T11:14:01.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:00 smithi062 ceph-mon[29666]: from='osd.6 [v2:172.21.15.139:6816/2768426846,v1:172.21.15.139:6817/2768426846]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:14:01.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:00 smithi062 ceph-mon[29666]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:14:01.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:00 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:01.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:00 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:14:01.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:00 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:14:01.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:00 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:14:01.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:00 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:14:01.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:00 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:14:01.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:00 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:14:01.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:00 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:01.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:00 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:02.258 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:01 smithi139 ceph-mon[25719]: Adjusting osd_memory_target on smithi139 to 5354M 2024-03-24T11:14:02.258 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:01 smithi139 ceph-mon[25719]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-03-24T11:14:02.258 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:01 smithi139 ceph-mon[25719]: osdmap e46: 7 total, 6 up, 7 in 2024-03-24T11:14:02.258 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:01 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T11:14:02.258 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:01 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T11:14:02.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:01 smithi062 ceph-mon[24799]: Adjusting osd_memory_target on smithi139 to 5354M 2024-03-24T11:14:02.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:01 smithi062 ceph-mon[24799]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-03-24T11:14:02.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:01 smithi062 ceph-mon[24799]: osdmap e46: 7 total, 6 up, 7 in 2024-03-24T11:14:02.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:01 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T11:14:02.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:01 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T11:14:02.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:01 smithi062 ceph-mon[29666]: Adjusting osd_memory_target on smithi139 to 5354M 2024-03-24T11:14:02.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:01 smithi062 ceph-mon[29666]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-03-24T11:14:02.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:01 smithi062 ceph-mon[29666]: osdmap e46: 7 total, 6 up, 7 in 2024-03-24T11:14:02.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:01 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T11:14:02.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:01 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T11:14:02.935 INFO:teuthology.orchestra.run.smithi139.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-24T11:14:02.936 INFO:teuthology.orchestra.run.smithi139.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-24T11:14:02.936 INFO:teuthology.orchestra.run.smithi139.stderr: stderr: 10+0 records in 2024-03-24T11:14:02.936 INFO:teuthology.orchestra.run.smithi139.stderr:10+0 records out 2024-03-24T11:14:02.936 INFO:teuthology.orchestra.run.smithi139.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0122816 s, 854 MB/s 2024-03-24T11:14:02.936 INFO:teuthology.orchestra.run.smithi139.stderr:--> Zapping successful for: 2024-03-24T11:14:03.266 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:02 smithi139 ceph-mon[25719]: purged_snaps scrub starts 2024-03-24T11:14:03.266 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:02 smithi139 ceph-mon[25719]: purged_snaps scrub ok 2024-03-24T11:14:03.266 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:02 smithi139 ceph-mon[25719]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-03-24T11:14:03.266 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:02 smithi139 ceph-mon[25719]: osd.6 [v2:172.21.15.139:6816/2768426846,v1:172.21.15.139:6817/2768426846] boot 2024-03-24T11:14:03.266 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:02 smithi139 ceph-mon[25719]: osdmap e47: 7 total, 7 up, 7 in 2024-03-24T11:14:03.267 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:02 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T11:14:03.267 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:02 smithi139 ceph-mon[25719]: osdmap e48: 7 total, 7 up, 7 in 2024-03-24T11:14:03.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:02 smithi062 ceph-mon[24799]: purged_snaps scrub starts 2024-03-24T11:14:03.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:02 smithi062 ceph-mon[24799]: purged_snaps scrub ok 2024-03-24T11:14:03.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:02 smithi062 ceph-mon[24799]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-03-24T11:14:03.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:02 smithi062 ceph-mon[24799]: osd.6 [v2:172.21.15.139:6816/2768426846,v1:172.21.15.139:6817/2768426846] boot 2024-03-24T11:14:03.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:02 smithi062 ceph-mon[24799]: osdmap e47: 7 total, 7 up, 7 in 2024-03-24T11:14:03.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:02 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T11:14:03.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:02 smithi062 ceph-mon[24799]: osdmap e48: 7 total, 7 up, 7 in 2024-03-24T11:14:03.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:02 smithi062 ceph-mon[29666]: purged_snaps scrub starts 2024-03-24T11:14:03.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:02 smithi062 ceph-mon[29666]: purged_snaps scrub ok 2024-03-24T11:14:03.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:02 smithi062 ceph-mon[29666]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2024-03-24T11:14:03.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:02 smithi062 ceph-mon[29666]: osd.6 [v2:172.21.15.139:6816/2768426846,v1:172.21.15.139:6817/2768426846] boot 2024-03-24T11:14:03.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:02 smithi062 ceph-mon[29666]: osdmap e47: 7 total, 7 up, 7 in 2024-03-24T11:14:03.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:02 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T11:14:03.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:02 smithi062 ceph-mon[29666]: osdmap e48: 7 total, 7 up, 7 in 2024-03-24T11:14:03.593 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi139:vg_nvme/lv_1 2024-03-24T11:14:05.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:04 smithi139 ceph-mon[25719]: pgmap v153: 1 pgs: 1 remapped+peering; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2024-03-24T11:14:05.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:04 smithi139 ceph-mon[25719]: osdmap e49: 7 total, 7 up, 7 in 2024-03-24T11:14:05.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:04 smithi062 ceph-mon[24799]: pgmap v153: 1 pgs: 1 remapped+peering; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2024-03-24T11:14:05.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:04 smithi062 ceph-mon[24799]: osdmap e49: 7 total, 7 up, 7 in 2024-03-24T11:14:05.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:04 smithi062 ceph-mon[29666]: pgmap v153: 1 pgs: 1 remapped+peering; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2024-03-24T11:14:05.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:04 smithi062 ceph-mon[29666]: osdmap e49: 7 total, 7 up, 7 in 2024-03-24T11:14:06.010 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:05 smithi139 ceph-mon[25719]: osdmap e50: 7 total, 7 up, 7 in 2024-03-24T11:14:06.011 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:05 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T11:14:06.011 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:05 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T11:14:06.011 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:05 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:14:06.011 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:05 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:06.011 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:05 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:06.011 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:05 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:14:06.011 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:05 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:14:06.011 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:05 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:14:06.011 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:05 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:06.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:14:05 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.139 - - [24/Mar/2024:11:14:05] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:14:06.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:05 smithi062 ceph-mon[24799]: osdmap e50: 7 total, 7 up, 7 in 2024-03-24T11:14:06.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:05 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T11:14:06.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:05 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T11:14:06.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:05 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:14:06.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:05 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:06.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:05 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:06.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:05 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:14:06.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:05 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:14:06.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:05 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:14:06.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:05 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:06.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:05 smithi062 ceph-mon[29666]: osdmap e50: 7 total, 7 up, 7 in 2024-03-24T11:14:06.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:05 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T11:14:06.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:05 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T11:14:06.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:05 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:14:06.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:05 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:06.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:05 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:06.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:05 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:14:06.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:05 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:14:06.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:05 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:14:06.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:05 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:07.143 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:06 smithi139 ceph-mon[25719]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:14:07.143 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:06 smithi139 ceph-mon[25719]: pgmap v156: 1 pgs: 1 remapped+peering; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2024-03-24T11:14:07.143 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:06 smithi139 ceph-mon[25719]: Detected new or changed devices on smithi139 2024-03-24T11:14:07.143 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:06 smithi139 ceph-mon[25719]: pgmap v157: 1 pgs: 1 remapped+peering; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2024-03-24T11:14:07.143 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:06 smithi139 ceph-mon[25719]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-03-24T11:14:07.143 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:06 smithi139 ceph-mon[25719]: osdmap e51: 7 total, 7 up, 7 in 2024-03-24T11:14:07.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:06 smithi062 ceph-mon[24799]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:14:07.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:06 smithi062 ceph-mon[24799]: pgmap v156: 1 pgs: 1 remapped+peering; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2024-03-24T11:14:07.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:06 smithi062 ceph-mon[24799]: Detected new or changed devices on smithi139 2024-03-24T11:14:07.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:06 smithi062 ceph-mon[24799]: pgmap v157: 1 pgs: 1 remapped+peering; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2024-03-24T11:14:07.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:06 smithi062 ceph-mon[24799]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-03-24T11:14:07.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:06 smithi062 ceph-mon[24799]: osdmap e51: 7 total, 7 up, 7 in 2024-03-24T11:14:07.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:06 smithi062 ceph-mon[29666]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:14:07.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:06 smithi062 ceph-mon[29666]: pgmap v156: 1 pgs: 1 remapped+peering; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2024-03-24T11:14:07.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:06 smithi062 ceph-mon[29666]: Detected new or changed devices on smithi139 2024-03-24T11:14:07.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:06 smithi062 ceph-mon[29666]: pgmap v157: 1 pgs: 1 remapped+peering; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2024-03-24T11:14:07.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:06 smithi062 ceph-mon[29666]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-03-24T11:14:07.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:06 smithi062 ceph-mon[29666]: osdmap e51: 7 total, 7 up, 7 in 2024-03-24T11:14:08.477 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:14:08 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.139 - - [24/Mar/2024:11:14:08] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:14:09.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:09 smithi139 ceph-mon[25719]: pgmap v159: 1 pgs: 1 remapped+peering; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2024-03-24T11:14:09.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:09 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:09.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:09 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:09.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:09 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:14:09.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:09 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:14:09.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:09 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:14:09.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:09 smithi139 ceph-mon[25719]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-03-24T11:14:09.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:09 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:09.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:09 smithi139 ceph-mon[25719]: from='client.? 172.21.15.139:0/2111777761' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5422547c-7038-4654-807c-e118f84575f1"}]: dispatch 2024-03-24T11:14:09.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:09 smithi139 ceph-mon[25719]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5422547c-7038-4654-807c-e118f84575f1"}]: dispatch 2024-03-24T11:14:09.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:09 smithi139 ceph-mon[25719]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5422547c-7038-4654-807c-e118f84575f1"}]': finished 2024-03-24T11:14:09.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:09 smithi139 ceph-mon[25719]: osdmap e52: 8 total, 7 up, 8 in 2024-03-24T11:14:09.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:09 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T11:14:09.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:09 smithi062 ceph-mon[24799]: pgmap v159: 1 pgs: 1 remapped+peering; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2024-03-24T11:14:09.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:09 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:09.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:09 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:09.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:09 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:14:09.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:09 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:14:09.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:09 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:14:09.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:09 smithi062 ceph-mon[24799]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-03-24T11:14:09.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:09 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:09.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:09 smithi062 ceph-mon[24799]: from='client.? 172.21.15.139:0/2111777761' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5422547c-7038-4654-807c-e118f84575f1"}]: dispatch 2024-03-24T11:14:09.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:09 smithi062 ceph-mon[24799]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5422547c-7038-4654-807c-e118f84575f1"}]: dispatch 2024-03-24T11:14:09.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:09 smithi062 ceph-mon[24799]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5422547c-7038-4654-807c-e118f84575f1"}]': finished 2024-03-24T11:14:09.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:09 smithi062 ceph-mon[24799]: osdmap e52: 8 total, 7 up, 8 in 2024-03-24T11:14:09.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:09 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T11:14:09.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:09 smithi062 ceph-mon[29666]: pgmap v159: 1 pgs: 1 remapped+peering; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2024-03-24T11:14:09.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:09 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:09.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:09 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:09.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:09 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:14:09.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:09 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:14:09.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:09 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:14:09.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:09 smithi062 ceph-mon[29666]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-03-24T11:14:09.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:09 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:09.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:09 smithi062 ceph-mon[29666]: from='client.? 172.21.15.139:0/2111777761' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5422547c-7038-4654-807c-e118f84575f1"}]: dispatch 2024-03-24T11:14:09.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:09 smithi062 ceph-mon[29666]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5422547c-7038-4654-807c-e118f84575f1"}]: dispatch 2024-03-24T11:14:09.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:09 smithi062 ceph-mon[29666]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5422547c-7038-4654-807c-e118f84575f1"}]': finished 2024-03-24T11:14:09.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:09 smithi062 ceph-mon[29666]: osdmap e52: 8 total, 7 up, 8 in 2024-03-24T11:14:09.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:09 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T11:14:10.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:10 smithi139 ceph-mon[25719]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-03-24T11:14:10.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:10 smithi139 ceph-mon[25719]: Cluster is now healthy 2024-03-24T11:14:10.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:10 smithi139 ceph-mon[25719]: from='client.? 172.21.15.139:0/100316063' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T11:14:10.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:10 smithi062 ceph-mon[24799]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-03-24T11:14:10.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:10 smithi062 ceph-mon[24799]: Cluster is now healthy 2024-03-24T11:14:10.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:10 smithi062 ceph-mon[24799]: from='client.? 172.21.15.139:0/100316063' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T11:14:10.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:10 smithi062 ceph-mon[29666]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-03-24T11:14:10.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:10 smithi062 ceph-mon[29666]: Cluster is now healthy 2024-03-24T11:14:10.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:10 smithi062 ceph-mon[29666]: from='client.? 172.21.15.139:0/100316063' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T11:14:11.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:11 smithi139 ceph-mon[25719]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-03-24T11:14:11.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:11 smithi062 ceph-mon[24799]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-03-24T11:14:11.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:11 smithi062 ceph-mon[29666]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-03-24T11:14:11.977 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:14:11 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.62 - - [24/Mar/2024:11:14:11] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:14:12.601 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:12 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:12.601 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:12 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:12.601 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:12 smithi139 ceph-mon[25719]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-03-24T11:14:12.828 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:12 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:12.828 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:12 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:12.829 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:12 smithi062 ceph-mon[29666]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-03-24T11:14:12.829 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:12 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:12.829 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:12 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:12.829 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:12 smithi062 ceph-mon[24799]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-03-24T11:14:15.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:15 smithi139 ceph-mon[25719]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-03-24T11:14:15.444 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:15 smithi062 ceph-mon[24799]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-03-24T11:14:15.444 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:15 smithi062 ceph-mon[29666]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-03-24T11:14:16.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:16 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-24T11:14:16.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:16 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:14:16.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:16 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-24T11:14:16.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:16 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:14:16.607 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:16 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-24T11:14:16.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:16 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:14:17.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:17 smithi062 ceph-mon[24799]: Deploying daemon osd.7 on smithi139 2024-03-24T11:14:17.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:17 smithi062 ceph-mon[24799]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-03-24T11:14:17.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:17 smithi062 ceph-mon[29666]: Deploying daemon osd.7 on smithi139 2024-03-24T11:14:17.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:17 smithi062 ceph-mon[29666]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-03-24T11:14:17.529 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:17 smithi139 ceph-mon[25719]: Deploying daemon osd.7 on smithi139 2024-03-24T11:14:17.529 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:17 smithi139 ceph-mon[25719]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-03-24T11:14:18.951 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:18 smithi139 ceph-mon[25719]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2024-03-24T11:14:19.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:18 smithi062 ceph-mon[24799]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2024-03-24T11:14:19.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:18 smithi062 ceph-mon[29666]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2024-03-24T11:14:21.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:21 smithi062 ceph-mon[24799]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2024-03-24T11:14:21.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:21 smithi062 ceph-mon[29666]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2024-03-24T11:14:21.539 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:21 smithi139 ceph-mon[25719]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2024-03-24T11:14:23.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:23 smithi062 ceph-mon[24799]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2024-03-24T11:14:23.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:23 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:23.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:23 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:14:23.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:23 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:14:23.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:23 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:14:23.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:23 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:23.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:23 smithi062 ceph-mon[29666]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2024-03-24T11:14:23.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:23 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:23.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:23 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:14:23.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:23 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:14:23.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:23 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:14:23.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:23 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:23.483 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:23 smithi139 ceph-mon[25719]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2024-03-24T11:14:23.483 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:23 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:23.483 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:23 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:14:23.483 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:23 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:14:23.483 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:23 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:14:23.483 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:23 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:24.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:24 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-24T11:14:24.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:24 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-24T11:14:24.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:24 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-24T11:14:24.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:24 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-24T11:14:24.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:24 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-24T11:14:24.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:24 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-24T11:14:24.881 INFO:teuthology.orchestra.run.smithi139.stdout:Created osd(s) 7 on host 'smithi139' 2024-03-24T11:14:25.435 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:25 smithi139 ceph-mon[25719]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2024-03-24T11:14:25.435 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:25 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:25.435 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:25 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:14:25.435 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:25 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:14:25.435 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:25 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:14:25.435 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:25 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:14:25.436 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:25 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:14:25.436 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:25 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:14:25.436 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:25 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:14:25.436 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:25 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:25.436 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:25 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:25.457 DEBUG:teuthology.orchestra.run.smithi139:osd.7> sudo journalctl -f -n 0 -u ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.7.service 2024-03-24T11:14:25.459 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-03-24T11:14:25.459 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph osd stat -f json 2024-03-24T11:14:25.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:25 smithi062 ceph-mon[24799]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2024-03-24T11:14:25.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:25 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:25.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:25 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:14:25.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:25 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:14:25.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:25 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:14:25.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:25 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:14:25.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:25 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:14:25.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:25 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:14:25.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:25 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:14:25.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:25 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:25.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:25 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:25.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:25 smithi062 ceph-mon[29666]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2024-03-24T11:14:25.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:25 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:25.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:25 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:14:25.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:25 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:14:25.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:25 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:14:25.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:25 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:14:25.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:25 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:14:25.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:25 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:14:25.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:25 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:14:25.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:25 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:25.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:25 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:26.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:26 smithi139 ceph-mon[25719]: Adjusting osd_memory_target on smithi139 to 4016M 2024-03-24T11:14:26.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:26 smithi139 ceph-mon[25719]: from='osd.7 [v2:172.21.15.139:6824/2297684619,v1:172.21.15.139:6825/2297684619]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-24T11:14:26.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:26 smithi139 ceph-mon[25719]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-24T11:14:26.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:26 smithi062 ceph-mon[24799]: Adjusting osd_memory_target on smithi139 to 4016M 2024-03-24T11:14:26.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:26 smithi062 ceph-mon[24799]: from='osd.7 [v2:172.21.15.139:6824/2297684619,v1:172.21.15.139:6825/2297684619]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-24T11:14:26.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:26 smithi062 ceph-mon[24799]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-24T11:14:26.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:26 smithi062 ceph-mon[29666]: Adjusting osd_memory_target on smithi139 to 4016M 2024-03-24T11:14:26.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:26 smithi062 ceph-mon[29666]: from='osd.7 [v2:172.21.15.139:6824/2297684619,v1:172.21.15.139:6825/2297684619]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-24T11:14:26.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:26 smithi062 ceph-mon[29666]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-24T11:14:27.342 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:27 smithi062 ceph-mon[29666]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2024-03-24T11:14:27.342 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:27 smithi062 ceph-mon[29666]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-24T11:14:27.342 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:27 smithi062 ceph-mon[29666]: osdmap e53: 8 total, 7 up, 8 in 2024-03-24T11:14:27.342 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:27 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T11:14:27.342 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:27 smithi062 ceph-mon[29666]: from='osd.7 [v2:172.21.15.139:6824/2297684619,v1:172.21.15.139:6825/2297684619]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:14:27.342 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:27 smithi062 ceph-mon[29666]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:14:27.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:27 smithi062 ceph-mon[24799]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2024-03-24T11:14:27.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:27 smithi062 ceph-mon[24799]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-24T11:14:27.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:27 smithi062 ceph-mon[24799]: osdmap e53: 8 total, 7 up, 8 in 2024-03-24T11:14:27.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:27 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T11:14:27.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:27 smithi062 ceph-mon[24799]: from='osd.7 [v2:172.21.15.139:6824/2297684619,v1:172.21.15.139:6825/2297684619]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:14:27.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:27 smithi062 ceph-mon[24799]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:14:27.346 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-24T11:14:27.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:27 smithi139 ceph-mon[25719]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2024-03-24T11:14:27.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:27 smithi139 ceph-mon[25719]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-24T11:14:27.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:27 smithi139 ceph-mon[25719]: osdmap e53: 8 total, 7 up, 8 in 2024-03-24T11:14:27.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:27 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T11:14:27.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:27 smithi139 ceph-mon[25719]: from='osd.7 [v2:172.21.15.139:6824/2297684619,v1:172.21.15.139:6825/2297684619]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:14:27.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:27 smithi139 ceph-mon[25719]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:14:27.609 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:14:27 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[39163]: 2024-03-24T11:14:27.181+0000 7f200deea700 -1 osd.7 0 waiting for initial osdmap 2024-03-24T11:14:27.609 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:14:27 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[39163]: 2024-03-24T11:14:27.189+0000 7f2008881700 -1 osd.7 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T11:14:27.726 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:14:27 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.139 - - [24/Mar/2024:11:14:27] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:14:27.955 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":54,"num_osds":8,"num_up_osds":7,"osd_up_since":1711278841,"num_in_osds":8,"osd_in_since":1711278848,"num_remapped_pgs":0} 2024-03-24T11:14:28.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:28 smithi062 ceph-mon[24799]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-03-24T11:14:28.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:28 smithi062 ceph-mon[24799]: osdmap e54: 8 total, 7 up, 8 in 2024-03-24T11:14:28.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:28 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T11:14:28.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:28 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T11:14:28.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:28 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:28.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:28 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:28.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:28 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/1972198655' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-24T11:14:28.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:28 smithi062 ceph-mon[24799]: osd.7 [v2:172.21.15.139:6824/2297684619,v1:172.21.15.139:6825/2297684619] boot 2024-03-24T11:14:28.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:28 smithi062 ceph-mon[24799]: osdmap e55: 8 total, 8 up, 8 in 2024-03-24T11:14:28.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:28 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T11:14:28.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:28 smithi062 ceph-mon[29666]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-03-24T11:14:28.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:28 smithi062 ceph-mon[29666]: osdmap e54: 8 total, 7 up, 8 in 2024-03-24T11:14:28.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:28 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T11:14:28.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:28 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T11:14:28.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:28 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:28.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:28 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:28.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:28 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/1972198655' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-24T11:14:28.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:28 smithi062 ceph-mon[29666]: osd.7 [v2:172.21.15.139:6824/2297684619,v1:172.21.15.139:6825/2297684619] boot 2024-03-24T11:14:28.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:28 smithi062 ceph-mon[29666]: osdmap e55: 8 total, 8 up, 8 in 2024-03-24T11:14:28.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:28 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T11:14:28.607 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:28 smithi139 ceph-mon[25719]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-03-24T11:14:28.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:28 smithi139 ceph-mon[25719]: osdmap e54: 8 total, 7 up, 8 in 2024-03-24T11:14:28.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:28 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T11:14:28.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:28 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T11:14:28.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:28 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:28.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:28 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:28.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:28 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/1972198655' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-24T11:14:28.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:28 smithi139 ceph-mon[25719]: osd.7 [v2:172.21.15.139:6824/2297684619,v1:172.21.15.139:6825/2297684619] boot 2024-03-24T11:14:28.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:28 smithi139 ceph-mon[25719]: osdmap e55: 8 total, 8 up, 8 in 2024-03-24T11:14:28.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:28 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T11:14:28.957 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph osd stat -f json 2024-03-24T11:14:29.279 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:29 smithi062 ceph-mon[24799]: purged_snaps scrub starts 2024-03-24T11:14:29.279 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:29 smithi062 ceph-mon[24799]: purged_snaps scrub ok 2024-03-24T11:14:29.279 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:29 smithi062 ceph-mon[24799]: Detected new or changed devices on smithi139 2024-03-24T11:14:29.279 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:29 smithi062 ceph-mon[24799]: pgmap v174: 1 pgs: 1 peering; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2024-03-24T11:14:29.279 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:29 smithi062 ceph-mon[24799]: osdmap e56: 8 total, 8 up, 8 in 2024-03-24T11:14:29.280 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:29 smithi062 ceph-mon[29666]: purged_snaps scrub starts 2024-03-24T11:14:29.280 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:29 smithi062 ceph-mon[29666]: purged_snaps scrub ok 2024-03-24T11:14:29.280 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:29 smithi062 ceph-mon[29666]: Detected new or changed devices on smithi139 2024-03-24T11:14:29.280 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:29 smithi062 ceph-mon[29666]: pgmap v174: 1 pgs: 1 peering; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2024-03-24T11:14:29.280 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:29 smithi062 ceph-mon[29666]: osdmap e56: 8 total, 8 up, 8 in 2024-03-24T11:14:29.607 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:29 smithi139 ceph-mon[25719]: purged_snaps scrub starts 2024-03-24T11:14:29.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:29 smithi139 ceph-mon[25719]: purged_snaps scrub ok 2024-03-24T11:14:29.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:29 smithi139 ceph-mon[25719]: Detected new or changed devices on smithi139 2024-03-24T11:14:29.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:29 smithi139 ceph-mon[25719]: pgmap v174: 1 pgs: 1 peering; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2024-03-24T11:14:29.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:29 smithi139 ceph-mon[25719]: osdmap e56: 8 total, 8 up, 8 in 2024-03-24T11:14:30.845 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-24T11:14:31.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:30 smithi139 ceph-mon[25719]: osdmap e57: 8 total, 8 up, 8 in 2024-03-24T11:14:31.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:30 smithi139 ceph-mon[25719]: pgmap v177: 1 pgs: 1 peering; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:14:31.226 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:30 smithi062 ceph-mon[29666]: osdmap e57: 8 total, 8 up, 8 in 2024-03-24T11:14:31.226 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:30 smithi062 ceph-mon[29666]: pgmap v177: 1 pgs: 1 peering; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:14:31.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:30 smithi062 ceph-mon[24799]: osdmap e57: 8 total, 8 up, 8 in 2024-03-24T11:14:31.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:30 smithi062 ceph-mon[24799]: pgmap v177: 1 pgs: 1 peering; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:14:31.540 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":57,"num_osds":8,"num_up_osds":8,"osd_up_since":1711278867,"num_in_osds":8,"osd_in_since":1711278848,"num_remapped_pgs":1} 2024-03-24T11:14:31.541 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-24T11:14:31.769 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T11:14:31.859 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:14:31 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.62 - - [24/Mar/2024:11:14:31] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:14:32.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:31 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/1705370579' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-24T11:14:32.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:31 smithi139 ceph-mon[25719]: osdmap e58: 8 total, 8 up, 8 in 2024-03-24T11:14:32.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:31 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:32.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:31 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:32.158 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:31 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/1705370579' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-24T11:14:32.158 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:31 smithi062 ceph-mon[24799]: osdmap e58: 8 total, 8 up, 8 in 2024-03-24T11:14:32.158 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:31 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:32.158 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:31 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:32.158 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:31 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/1705370579' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-24T11:14:32.158 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:31 smithi062 ceph-mon[29666]: osdmap e58: 8 total, 8 up, 8 in 2024-03-24T11:14:32.159 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:31 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:32.159 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:31 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:33.082 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:32 smithi062 ceph-mon[24799]: osdmap e59: 8 total, 8 up, 8 in 2024-03-24T11:14:33.083 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:32 smithi062 ceph-mon[24799]: pgmap v180: 1 pgs: 1 peering; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:14:33.083 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:32 smithi062 ceph-mon[29666]: osdmap e59: 8 total, 8 up, 8 in 2024-03-24T11:14:33.083 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:32 smithi062 ceph-mon[29666]: pgmap v180: 1 pgs: 1 peering; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:14:33.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:32 smithi139 ceph-mon[25719]: osdmap e59: 8 total, 8 up, 8 in 2024-03-24T11:14:33.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:32 smithi139 ceph-mon[25719]: pgmap v180: 1 pgs: 1 peering; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:14:33.495 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-24T11:14:33.495 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":59,"fsid":"ebb89568-e9ce-11ee-95cd-87774f69a715","created":"2024-03-24T11:09:33.575451+0000","modified":"2024-03-24T11:14:31.848388+0000","last_up_change":"2024-03-24T11:14:27.827075+0000","last_in_change":"2024-03-24T11:14:08.484969+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-24T11:12:32.117868+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"83103b18-933d-437b-9762-583f85bc1776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1287363541},{"type":"v1","addr":"172.21.15.62:6803","nonce":1287363541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":1287363541},{"type":"v1","addr":"172.21.15.62:6805","nonce":1287363541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":1287363541},{"type":"v1","addr":"172.21.15.62:6809","nonce":1287363541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1287363541},{"type":"v1","addr":"172.21.15.62:6807","nonce":1287363541}]},"public_addr":"172.21.15.62:6803/1287363541","cluster_addr":"172.21.15.62:6805/1287363541","heartbeat_back_addr":"172.21.15.62:6809/1287363541","heartbeat_front_addr":"172.21.15.62:6807/1287363541","state":["exists","up"]},{"osd":1,"uuid":"515c2c9b-7bc4-4b00-9be7-8773cf551306","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1156951329},{"type":"v1","addr":"172.21.15.62:6811","nonce":1156951329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1156951329},{"type":"v1","addr":"172.21.15.62:6813","nonce":1156951329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1156951329},{"type":"v1","addr":"172.21.15.62:6817","nonce":1156951329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1156951329},{"type":"v1","addr":"172.21.15.62:6815","nonce":1156951329}]},"public_addr":"172.21.15.62:6811/1156951329","cluster_addr":"172.21.15.62:6813/1156951329","heartbeat_back_addr":"172.21.15.62:6817/1156951329","heartbeat_front_addr":"172.21.15.62:6815/1156951329","state":["exists","up"]},{"osd":2,"uuid":"569ff950-c8b0-48fc-b137-4598571ff36c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2474659851},{"type":"v1","addr":"172.21.15.62:6819","nonce":2474659851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2474659851},{"type":"v1","addr":"172.21.15.62:6821","nonce":2474659851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":2474659851},{"type":"v1","addr":"172.21.15.62:6825","nonce":2474659851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2474659851},{"type":"v1","addr":"172.21.15.62:6823","nonce":2474659851}]},"public_addr":"172.21.15.62:6819/2474659851","cluster_addr":"172.21.15.62:6821/2474659851","heartbeat_back_addr":"172.21.15.62:6825/2474659851","heartbeat_front_addr":"172.21.15.62:6823/2474659851","state":["exists","up"]},{"osd":3,"uuid":"e0c09510-dcba-4332-9c02-3842c992612b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":2104856662},{"type":"v1","addr":"172.21.15.62:6827","nonce":2104856662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":2104856662},{"type":"v1","addr":"172.21.15.62:6829","nonce":2104856662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2104856662},{"type":"v1","addr":"172.21.15.62:6833","nonce":2104856662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":2104856662},{"type":"v1","addr":"172.21.15.62:6831","nonce":2104856662}]},"public_addr":"172.21.15.62:6827/2104856662","cluster_addr":"172.21.15.62:6829/2104856662","heartbeat_back_addr":"172.21.15.62:6833/2104856662","heartbeat_front_addr":"172.21.15.62:6831/2104856662","state":["exists","up"]},{"osd":4,"uuid":"8c96358d-d11f-4f66-a433-3e4dc3024322","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3866478379},{"type":"v1","addr":"172.21.15.139:6801","nonce":3866478379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3866478379},{"type":"v1","addr":"172.21.15.139:6803","nonce":3866478379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3866478379},{"type":"v1","addr":"172.21.15.139:6807","nonce":3866478379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3866478379},{"type":"v1","addr":"172.21.15.139:6805","nonce":3866478379}]},"public_addr":"172.21.15.139:6801/3866478379","cluster_addr":"172.21.15.139:6803/3866478379","heartbeat_back_addr":"172.21.15.139:6807/3866478379","heartbeat_front_addr":"172.21.15.139:6805/3866478379","state":["exists","up"]},{"osd":5,"uuid":"adf8b0ea-4d03-4bc4-bca4-2b6aad9e967d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1384318027},{"type":"v1","addr":"172.21.15.139:6809","nonce":1384318027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1384318027},{"type":"v1","addr":"172.21.15.139:6811","nonce":1384318027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1384318027},{"type":"v1","addr":"172.21.15.139:6815","nonce":1384318027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1384318027},{"type":"v1","addr":"172.21.15.139:6813","nonce":1384318027}]},"public_addr":"172.21.15.139:6809/1384318027","cluster_addr":"172.21.15.139:6811/1384318027","heartbeat_back_addr":"172.21.15.139:6815/1384318027","heartbeat_front_addr":"172.21.15.139:6813/1384318027","state":["exists","up"]},{"osd":6,"uuid":"f9790a56-fa0c-4908-8fbc-c347ecefa2df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":2768426846},{"type":"v1","addr":"172.21.15.139:6817","nonce":2768426846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":2768426846},{"type":"v1","addr":"172.21.15.139:6819","nonce":2768426846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":2768426846},{"type":"v1","addr":"172.21.15.139:6823","nonce":2768426846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":2768426846},{"type":"v1","addr":"172.21.15.139:6821","nonce":2768426846}]},"public_addr":"172.21.15.139:6817/2768426846","cluster_addr":"172.21.15.139:6819/2768426846","heartbeat_back_addr":"172.21.15.139:6823/2768426846","heartbeat_front_addr":"172.21.15.139:6821/2768426846","state":["exists","up"]},{"osd":7,"uuid":"5422547c-7038-4654-807c-e118f84575f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2297684619},{"type":"v1","addr":"172.21.15.139:6825","nonce":2297684619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2297684619},{"type":"v1","addr":"172.21.15.139:6827","nonce":2297684619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2297684619},{"type":"v1","addr":"172.21.15.139:6831","nonce":2297684619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2297684619},{"type":"v1","addr":"172.21.15.139:6829","nonce":2297684619}]},"public_addr":"172.21.15.139:6825/2297684619","cluster_addr":"172.21.15.139:6827/2297684619","heartbeat_back_addr":"172.21.15.139:6831/2297684619","heartbeat_front_addr":"172.21.15.139:6829/2297684619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T11:11:44.756685+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T11:12:06.837947+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T11:12:28.819088+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T11:12:52.203208+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T11:13:13.838606+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T11:13:36.993722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T11:14:00.888906+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T11:14:26.280662+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/2609891207":"2024-03-25T11:10:23.129060+0000","172.21.15.62:0/2764833578":"2024-03-25T11:10:23.129060+0000","172.21.15.62:6800/2609891207":"2024-03-25T11:10:23.129060+0000","172.21.15.62:0/81242112":"2024-03-25T11:10:23.129060+0000","172.21.15.62:0/2039307015":"2024-03-25T11:10:00.759801+0000","172.21.15.62:0/4080145397":"2024-03-25T11:10:23.129060+0000","172.21.15.62:0/936493210":"2024-03-25T11:10:00.759801+0000","172.21.15.62:0/106493134":"2024-03-25T11:10:00.759801+0000","172.21.15.62:6800/1616593108":"2024-03-25T11:10:00.759801+0000","172.21.15.62:6801/1616593108":"2024-03-25T11:10:00.759801+0000"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-24T11:14:34.026 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:33 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/1197439760' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T11:14:34.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:33 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/1197439760' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T11:14:34.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:33 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/1197439760' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T11:14:34.620 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-24T11:12:32.117868+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-03-24T11:14:34.620 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-24T11:14:34.976 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:34 smithi062 ceph-mon[24799]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-03-24T11:14:34.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:34 smithi062 ceph-mon[29666]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-03-24T11:14:35.028 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T11:14:35.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:34 smithi139 ceph-mon[25719]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-03-24T11:14:37.091 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-03-24T11:14:37.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:37 smithi062 ceph-mon[24799]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-03-24T11:14:37.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:37 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/2981560869' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T11:14:37.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:37 smithi062 ceph-mon[29666]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-03-24T11:14:37.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:37 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/2981560869' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T11:14:37.607 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:37 smithi139 ceph-mon[25719]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-03-24T11:14:37.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:37 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/2981560869' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T11:14:37.995 INFO:tasks.cephadm:Adding prometheus.a on smithi139 2024-03-24T11:14:37.995 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph orch apply prometheus '1;smithi139=a' 2024-03-24T11:14:39.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:38 smithi139 ceph-mon[25719]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-03-24T11:14:39.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:38 smithi062 ceph-mon[24799]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-03-24T11:14:39.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:38 smithi062 ceph-mon[29666]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-03-24T11:14:39.831 INFO:teuthology.orchestra.run.smithi139.stdout:Scheduled prometheus update... 2024-03-24T11:14:40.473 DEBUG:teuthology.orchestra.run.smithi139:prometheus.a> sudo journalctl -f -n 0 -u ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@prometheus.a.service 2024-03-24T11:14:40.475 INFO:tasks.cephadm:Adding node-exporter.a on smithi062 2024-03-24T11:14:40.475 INFO:tasks.cephadm:Adding node-exporter.b on smithi139 2024-03-24T11:14:40.475 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph orch apply node-exporter '2;smithi062=a;smithi139=b' 2024-03-24T11:14:41.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:40 smithi139 ceph-mon[25719]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi139=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:14:41.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:40 smithi139 ceph-mon[25719]: Saving service prometheus spec with placement smithi139=a;count:1 2024-03-24T11:14:41.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:40 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:41.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:40 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:14:41.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:40 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:14:41.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:40 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:14:41.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:40 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:41.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:40 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-03-24T11:14:41.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:40 smithi139 ceph-mon[25719]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-03-24T11:14:41.109 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:14:40 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: ignoring --setuser ceph since I am not root 2024-03-24T11:14:41.109 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:14:40 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: ignoring --setgroup ceph since I am not root 2024-03-24T11:14:41.109 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:14:40 smithi139 ceph-mgr[27235]: -- 172.21.15.139:0/3268912385 <== mon.2 v2:172.21.15.139:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555a9dede340 con 0x555a9deb6800 2024-03-24T11:14:41.226 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:40 smithi062 ceph-mon[29666]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi139=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:14:41.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:40 smithi062 ceph-mon[29666]: Saving service prometheus spec with placement smithi139=a;count:1 2024-03-24T11:14:41.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:40 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:41.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:40 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:14:41.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:40 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:14:41.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:40 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:14:41.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:40 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:41.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:40 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-03-24T11:14:41.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:40 smithi062 ceph-mon[29666]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-03-24T11:14:41.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:14:40 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: ignoring --setuser ceph since I am not root 2024-03-24T11:14:41.228 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:14:40 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: ignoring --setgroup ceph since I am not root 2024-03-24T11:14:41.228 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:14:40 smithi062 ceph-mgr[25046]: -- 172.21.15.62:0/1344258151 <== mon.1 v2:172.21.15.62:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555acfad2340 con 0x555acfaaa800 2024-03-24T11:14:41.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:40 smithi062 ceph-mon[24799]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi139=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:14:41.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:40 smithi062 ceph-mon[24799]: Saving service prometheus spec with placement smithi139=a;count:1 2024-03-24T11:14:41.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:40 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:41.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:40 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:14:41.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:40 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:14:41.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:40 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:14:41.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:40 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' 2024-03-24T11:14:41.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:40 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-03-24T11:14:41.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:40 smithi062 ceph-mon[24799]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-03-24T11:14:41.654 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:14:41 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: 2024-03-24T11:14:41.364+0000 7f444866f000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-24T11:14:41.654 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:14:41 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: 2024-03-24T11:14:41.456+0000 7f444866f000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-24T11:14:41.726 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:14:41 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:14:41.379+0000 7f6c45ef0000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-24T11:14:41.727 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:14:41 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:14:41.471+0000 7f6c45ef0000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-24T11:14:41.918 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:41 smithi139 ceph-mon[25719]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-03-24T11:14:41.919 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:41 smithi139 ceph-mon[25719]: mgrmap e16: y(active, since 4m), standbys: x 2024-03-24T11:14:42.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:41 smithi062 ceph-mon[24799]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-03-24T11:14:42.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:41 smithi062 ceph-mon[24799]: mgrmap e16: y(active, since 4m), standbys: x 2024-03-24T11:14:42.172 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:14:42 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:14:42.080+0000 7f6c45ef0000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-24T11:14:42.172 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:41 smithi062 ceph-mon[29666]: from='mgr.14150 172.21.15.62:0/3992017221' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-03-24T11:14:42.172 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:41 smithi062 ceph-mon[29666]: mgrmap e16: y(active, since 4m), standbys: x 2024-03-24T11:14:42.242 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:14:42 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: 2024-03-24T11:14:42.064+0000 7f444866f000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-24T11:14:42.243 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:14:42 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: 2024-03-24T11:14:42.157+0000 7f444866f000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-24T11:14:42.243 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:14:42 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: 2024-03-24T11:14:42.242+0000 7f444866f000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-24T11:14:42.476 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:14:42 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:14:42.171+0000 7f6c45ef0000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-24T11:14:42.476 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:14:42 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:14:42.256+0000 7f6c45ef0000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-24T11:14:42.477 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:14:42 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:14:42.408+0000 7f6c45ef0000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-24T11:14:42.581 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:14:42 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: 2024-03-24T11:14:42.393+0000 7f444866f000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-24T11:14:42.581 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:14:42 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: 2024-03-24T11:14:42.474+0000 7f444866f000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-24T11:14:42.581 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:14:42 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: 2024-03-24T11:14:42.581+0000 7f444866f000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-24T11:14:42.853 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:14:42 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:14:42.490+0000 7f6c45ef0000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-24T11:14:42.853 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:14:42 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:14:42.602+0000 7f6c45ef0000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-24T11:14:42.853 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:14:42 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:14:42.687+0000 7f6c45ef0000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-24T11:14:42.857 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:14:42 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: 2024-03-24T11:14:42.664+0000 7f444866f000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-24T11:14:42.857 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:14:42 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: 2024-03-24T11:14:42.828+0000 7f444866f000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-24T11:14:43.226 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:14:42 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:14:42.853+0000 7f6c45ef0000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-24T11:14:43.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:14:42 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:14:42.940+0000 7f6c45ef0000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-24T11:14:43.357 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:14:42 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: 2024-03-24T11:14:42.914+0000 7f444866f000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-24T11:14:44.253 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:14:43 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: 2024-03-24T11:14:43.929+0000 7f444866f000 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-24T11:14:44.253 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:14:44 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: 2024-03-24T11:14:44.063+0000 7f444866f000 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-24T11:14:44.317 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:14:43 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:14:43.988+0000 7f6c45ef0000 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-24T11:14:44.317 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:14:44 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:14:44.121+0000 7f6c45ef0000 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-24T11:14:44.593 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:14:44 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: 2024-03-24T11:14:44.252+0000 7f444866f000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-24T11:14:44.594 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:14:44 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: 2024-03-24T11:14:44.340+0000 7f444866f000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-24T11:14:44.594 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:14:44 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: 2024-03-24T11:14:44.475+0000 7f444866f000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-24T11:14:44.727 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:14:44 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:14:44.317+0000 7f6c45ef0000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-24T11:14:44.727 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:14:44 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:14:44.406+0000 7f6c45ef0000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-24T11:14:44.727 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:14:44 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:14:44.546+0000 7f6c45ef0000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-24T11:14:44.857 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:14:44 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: 2024-03-24T11:14:44.801+0000 7f444866f000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-24T11:14:45.186 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:14:44 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:14:44.871+0000 7f6c45ef0000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-24T11:14:45.186 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:14:45 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:14:45.098+0000 7f6c45ef0000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-24T11:14:45.186 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:14:45 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:14:45.185+0000 7f6c45ef0000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-24T11:14:45.273 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:14:45 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: 2024-03-24T11:14:45.020+0000 7f444866f000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-24T11:14:45.274 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:14:45 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: 2024-03-24T11:14:45.104+0000 7f444866f000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-24T11:14:45.274 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:14:45 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: 2024-03-24T11:14:45.273+0000 7f444866f000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-24T11:14:45.476 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:14:45 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:14:45.360+0000 7f6c45ef0000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-24T11:14:46.485 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:14:46 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: 2024-03-24T11:14:46.184+0000 7f444866f000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-24T11:14:46.610 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:14:46 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:14:46.304+0000 7f6c45ef0000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-24T11:14:46.741 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:14:46 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: 2024-03-24T11:14:46.495+0000 7f444866f000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-24T11:14:46.742 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:14:46 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: 2024-03-24T11:14:46.591+0000 7f444866f000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-24T11:14:46.742 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:14:46 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: 2024-03-24T11:14:46.741+0000 7f444866f000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-24T11:14:46.867 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:46 smithi062 ceph-mon[24799]: Standby manager daemon x restarted 2024-03-24T11:14:46.867 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:46 smithi062 ceph-mon[24799]: Standby manager daemon x started 2024-03-24T11:14:46.868 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:46 smithi062 ceph-mon[24799]: from='mgr.? 172.21.15.139:0/2972222294' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-24T11:14:46.868 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:46 smithi062 ceph-mon[24799]: from='mgr.? 172.21.15.139:0/2972222294' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T11:14:46.868 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:46 smithi062 ceph-mon[24799]: from='mgr.? 172.21.15.139:0/2972222294' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-24T11:14:46.868 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:46 smithi062 ceph-mon[24799]: from='mgr.? 172.21.15.139:0/2972222294' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T11:14:46.868 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:14:46 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:14:46.609+0000 7f6c45ef0000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-24T11:14:46.868 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:14:46 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:14:46.708+0000 7f6c45ef0000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-24T11:14:46.868 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:14:46 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:14:46.867+0000 7f6c45ef0000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-24T11:14:46.868 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:46 smithi062 ceph-mon[29666]: Standby manager daemon x restarted 2024-03-24T11:14:46.869 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:46 smithi062 ceph-mon[29666]: Standby manager daemon x started 2024-03-24T11:14:46.869 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:46 smithi062 ceph-mon[29666]: from='mgr.? 172.21.15.139:0/2972222294' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-24T11:14:46.869 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:46 smithi062 ceph-mon[29666]: from='mgr.? 172.21.15.139:0/2972222294' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T11:14:46.869 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:46 smithi062 ceph-mon[29666]: from='mgr.? 172.21.15.139:0/2972222294' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-24T11:14:46.869 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:46 smithi062 ceph-mon[29666]: from='mgr.? 172.21.15.139:0/2972222294' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T11:14:47.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:46 smithi139 ceph-mon[25719]: Standby manager daemon x restarted 2024-03-24T11:14:47.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:46 smithi139 ceph-mon[25719]: Standby manager daemon x started 2024-03-24T11:14:47.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:46 smithi139 ceph-mon[25719]: from='mgr.? 172.21.15.139:0/2972222294' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-24T11:14:47.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:46 smithi139 ceph-mon[25719]: from='mgr.? 172.21.15.139:0/2972222294' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T11:14:47.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:46 smithi139 ceph-mon[25719]: from='mgr.? 172.21.15.139:0/2972222294' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-24T11:14:47.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:46 smithi139 ceph-mon[25719]: from='mgr.? 172.21.15.139:0/2972222294' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T11:14:47.108 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:14:46 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: [24/Mar/2024:11:14:46] ENGINE Bus STARTING 2024-03-24T11:14:47.108 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:14:46 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: CherryPy Checker: 2024-03-24T11:14:47.108 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:14:46 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: The Application mounted at '' has an empty config. 2024-03-24T11:14:47.109 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:14:46 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: 2024-03-24T11:14:47.109 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:14:46 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: [24/Mar/2024:11:14:46] ENGINE Serving on http://:::9283 2024-03-24T11:14:47.109 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:14:46 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: [24/Mar/2024:11:14:46] ENGINE Bus STARTED 2024-03-24T11:14:48.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:47 smithi139 ceph-mon[25719]: mgrmap e17: y(active, since 4m), standbys: x 2024-03-24T11:14:48.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:47 smithi139 ceph-mon[25719]: Active manager daemon y restarted 2024-03-24T11:14:48.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:47 smithi139 ceph-mon[25719]: Activating manager daemon y 2024-03-24T11:14:48.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:47 smithi139 ceph-mon[25719]: osdmap e60: 8 total, 8 up, 8 in 2024-03-24T11:14:48.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:47 smithi062 ceph-mon[24799]: mgrmap e17: y(active, since 4m), standbys: x 2024-03-24T11:14:48.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:47 smithi062 ceph-mon[24799]: Active manager daemon y restarted 2024-03-24T11:14:48.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:47 smithi062 ceph-mon[24799]: Activating manager daemon y 2024-03-24T11:14:48.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:47 smithi062 ceph-mon[24799]: osdmap e60: 8 total, 8 up, 8 in 2024-03-24T11:14:48.173 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:14:47 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: [24/Mar/2024:11:14:47] ENGINE Bus STARTING 2024-03-24T11:14:48.173 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:47 smithi062 ceph-mon[29666]: mgrmap e17: y(active, since 4m), standbys: x 2024-03-24T11:14:48.173 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:47 smithi062 ceph-mon[29666]: Active manager daemon y restarted 2024-03-24T11:14:48.173 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:47 smithi062 ceph-mon[29666]: Activating manager daemon y 2024-03-24T11:14:48.173 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:47 smithi062 ceph-mon[29666]: osdmap e60: 8 total, 8 up, 8 in 2024-03-24T11:14:48.431 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: CherryPy Checker: 2024-03-24T11:14:48.431 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: The Application mounted at '' has an empty config. 2024-03-24T11:14:48.431 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:14:48.431 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: [24/Mar/2024:11:14:48] ENGINE Serving on http://:::9283 2024-03-24T11:14:48.431 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: [24/Mar/2024:11:14:48] ENGINE Bus STARTED 2024-03-24T11:14:48.431 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: [24/Mar/2024:11:14:48] ENGINE Bus STARTING 2024-03-24T11:14:48.682 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: [24/Mar/2024:11:14:48] ENGINE Serving on https://172.21.15.62:7150 2024-03-24T11:14:48.682 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: [24/Mar/2024:11:14:48] ENGINE Bus STARTED 2024-03-24T11:14:48.855 INFO:teuthology.orchestra.run.smithi139.stdout:Scheduled node-exporter update... 2024-03-24T11:14:48.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[24799]: mgrmap e18: y(active, starting, since 0.947617s), standbys: x 2024-03-24T11:14:48.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T11:14:48.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T11:14:48.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:14:48.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-24T11:14:48.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-24T11:14:48.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T11:14:48.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T11:14:48.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T11:14:48.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T11:14:48.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T11:14:48.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T11:14:48.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T11:14:48.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T11:14:48.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-24T11:14:48.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-24T11:14:48.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-24T11:14:48.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[24799]: Manager daemon y is now available 2024-03-24T11:14:48.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:14:48.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:14:48.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:14:48.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:14:48.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:14:48.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:14:48.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi062", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:14:48.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi139", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:14:48.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi062", "name": "osd_memory_target"}]': finished 2024-03-24T11:14:48.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[29666]: mgrmap e18: y(active, starting, since 0.947617s), standbys: x 2024-03-24T11:14:48.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T11:14:48.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T11:14:48.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:14:48.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-24T11:14:48.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-24T11:14:48.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T11:14:48.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T11:14:48.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T11:14:48.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T11:14:48.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T11:14:48.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T11:14:48.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T11:14:48.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T11:14:48.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-24T11:14:48.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-24T11:14:48.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-24T11:14:48.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[29666]: Manager daemon y is now available 2024-03-24T11:14:48.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:14:48.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:14:48.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:14:48.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:14:48.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:14:48.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:14:48.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi062", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:14:48.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi139", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:14:48.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi062", "name": "osd_memory_target"}]': finished 2024-03-24T11:14:48.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[24799]: Updating smithi062:/etc/ceph/ceph.conf 2024-03-24T11:14:48.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi139", "name": "osd_memory_target"}]': finished 2024-03-24T11:14:48.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[24799]: Updating smithi139:/etc/ceph/ceph.conf 2024-03-24T11:14:48.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-24T11:14:48.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-24T11:14:48.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:14:48.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[29666]: Updating smithi062:/etc/ceph/ceph.conf 2024-03-24T11:14:48.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi139", "name": "osd_memory_target"}]': finished 2024-03-24T11:14:48.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[29666]: Updating smithi139:/etc/ceph/ceph.conf 2024-03-24T11:14:48.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-24T11:14:48.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-24T11:14:48.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:48 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:14:49.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:48 smithi139 ceph-mon[25719]: mgrmap e18: y(active, starting, since 0.947617s), standbys: x 2024-03-24T11:14:49.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:48 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T11:14:49.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:48 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T11:14:49.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:48 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:14:49.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:48 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-24T11:14:49.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:48 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-24T11:14:49.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:48 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T11:14:49.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:48 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T11:14:49.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:48 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T11:14:49.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:48 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T11:14:49.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:48 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T11:14:49.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:48 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T11:14:49.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:48 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T11:14:49.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:48 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T11:14:49.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:48 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-24T11:14:49.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:48 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-24T11:14:49.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:48 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-24T11:14:49.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:48 smithi139 ceph-mon[25719]: Manager daemon y is now available 2024-03-24T11:14:49.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:48 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:14:49.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:48 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:14:49.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:48 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:14:49.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:48 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:14:49.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:48 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:14:49.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:48 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:14:49.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:48 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi062", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:14:49.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:48 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi139", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:14:49.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:48 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi062", "name": "osd_memory_target"}]': finished 2024-03-24T11:14:49.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:48 smithi139 ceph-mon[25719]: Updating smithi062:/etc/ceph/ceph.conf 2024-03-24T11:14:49.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:48 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi139", "name": "osd_memory_target"}]': finished 2024-03-24T11:14:49.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:48 smithi139 ceph-mon[25719]: Updating smithi139:/etc/ceph/ceph.conf 2024-03-24T11:14:49.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:48 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-24T11:14:49.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:48 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-24T11:14:49.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:48 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:14:49.471 DEBUG:teuthology.orchestra.run.smithi062:node-exporter.a> sudo journalctl -f -n 0 -u ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@node-exporter.a.service 2024-03-24T11:14:49.474 DEBUG:teuthology.orchestra.run.smithi139:node-exporter.b> sudo journalctl -f -n 0 -u ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@node-exporter.b.service 2024-03-24T11:14:49.476 INFO:tasks.cephadm:Adding alertmanager.a on smithi062 2024-03-24T11:14:49.477 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph orch apply alertmanager '1;smithi062=a' 2024-03-24T11:14:49.971 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:49 smithi139 ceph-mon[25719]: [24/Mar/2024:11:14:48] ENGINE Bus STARTING 2024-03-24T11:14:49.971 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:49 smithi139 ceph-mon[25719]: [24/Mar/2024:11:14:48] ENGINE Serving on https://172.21.15.62:7150 2024-03-24T11:14:49.971 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:49 smithi139 ceph-mon[25719]: [24/Mar/2024:11:14:48] ENGINE Bus STARTED 2024-03-24T11:14:49.971 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:49 smithi139 ceph-mon[25719]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2024-03-24T11:14:49.971 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:49 smithi139 ceph-mon[25719]: Updating smithi139:/etc/ceph/ceph.client.admin.keyring 2024-03-24T11:14:49.971 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:49 smithi139 ceph-mon[25719]: mgrmap e19: y(active, since 1.96272s), standbys: x 2024-03-24T11:14:49.971 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:49 smithi139 ceph-mon[25719]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi062=a;smithi139=b", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:14:49.971 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:49 smithi139 ceph-mon[25719]: pgmap v3: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:14:49.971 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:49 smithi139 ceph-mon[25719]: Saving service node-exporter spec with placement smithi062=a;smithi139=b;count:2 2024-03-24T11:14:49.971 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:49 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:14:49.971 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:49 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:14:49.972 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:49 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:14:49.972 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:49 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:14:49.972 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:49 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-03-24T11:14:50.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:49 smithi062 ceph-mon[24799]: [24/Mar/2024:11:14:48] ENGINE Bus STARTING 2024-03-24T11:14:50.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:49 smithi062 ceph-mon[24799]: [24/Mar/2024:11:14:48] ENGINE Serving on https://172.21.15.62:7150 2024-03-24T11:14:50.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:49 smithi062 ceph-mon[24799]: [24/Mar/2024:11:14:48] ENGINE Bus STARTED 2024-03-24T11:14:50.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:49 smithi062 ceph-mon[24799]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2024-03-24T11:14:50.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:49 smithi062 ceph-mon[24799]: Updating smithi139:/etc/ceph/ceph.client.admin.keyring 2024-03-24T11:14:50.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:49 smithi062 ceph-mon[24799]: mgrmap e19: y(active, since 1.96272s), standbys: x 2024-03-24T11:14:50.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:49 smithi062 ceph-mon[24799]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi062=a;smithi139=b", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:14:50.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:49 smithi062 ceph-mon[24799]: pgmap v3: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:14:50.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:49 smithi062 ceph-mon[24799]: Saving service node-exporter spec with placement smithi062=a;smithi139=b;count:2 2024-03-24T11:14:50.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:49 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:14:50.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:49 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:14:50.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:49 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:14:50.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:49 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:14:50.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:49 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-03-24T11:14:50.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:49 smithi062 ceph-mon[29666]: [24/Mar/2024:11:14:48] ENGINE Bus STARTING 2024-03-24T11:14:50.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:49 smithi062 ceph-mon[29666]: [24/Mar/2024:11:14:48] ENGINE Serving on https://172.21.15.62:7150 2024-03-24T11:14:50.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:49 smithi062 ceph-mon[29666]: [24/Mar/2024:11:14:48] ENGINE Bus STARTED 2024-03-24T11:14:50.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:49 smithi062 ceph-mon[29666]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2024-03-24T11:14:50.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:49 smithi062 ceph-mon[29666]: Updating smithi139:/etc/ceph/ceph.client.admin.keyring 2024-03-24T11:14:50.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:49 smithi062 ceph-mon[29666]: mgrmap e19: y(active, since 1.96272s), standbys: x 2024-03-24T11:14:50.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:49 smithi062 ceph-mon[29666]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi062=a;smithi139=b", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:14:50.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:49 smithi062 ceph-mon[29666]: pgmap v3: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:14:50.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:49 smithi062 ceph-mon[29666]: Saving service node-exporter spec with placement smithi062=a;smithi139=b;count:2 2024-03-24T11:14:50.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:49 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:14:50.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:49 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:14:50.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:49 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:14:50.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:49 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:14:50.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:49 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-03-24T11:14:51.163 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:50 smithi139 ceph-mon[25719]: pgmap v4: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:14:51.163 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:50 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-03-24T11:14:51.163 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:50 smithi139 ceph-mon[25719]: mgrmap e20: y(active, since 3s), standbys: x 2024-03-24T11:14:51.163 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:50 smithi139 ceph-mon[25719]: Deploying daemon prometheus.a on smithi139 2024-03-24T11:14:51.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:50 smithi062 ceph-mon[24799]: pgmap v4: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:14:51.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:50 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-03-24T11:14:51.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:50 smithi062 ceph-mon[24799]: mgrmap e20: y(active, since 3s), standbys: x 2024-03-24T11:14:51.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:50 smithi062 ceph-mon[24799]: Deploying daemon prometheus.a on smithi139 2024-03-24T11:14:51.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:50 smithi062 ceph-mon[29666]: pgmap v4: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:14:51.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:50 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-03-24T11:14:51.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:50 smithi062 ceph-mon[29666]: mgrmap e20: y(active, since 3s), standbys: x 2024-03-24T11:14:51.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:50 smithi062 ceph-mon[29666]: Deploying daemon prometheus.a on smithi139 2024-03-24T11:14:52.226 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:14:51 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.62 - - [24/Mar/2024:11:14:51] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:14:52.458 INFO:teuthology.orchestra.run.smithi139.stdout:Scheduled alertmanager update... 2024-03-24T11:14:53.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:52 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:14:53.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:52 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:14:53.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:52 smithi139 ceph-mon[25719]: pgmap v5: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:14:53.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:52 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:14:53.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:52 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:14:53.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:52 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:14:53.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:52 smithi062 ceph-mon[24799]: pgmap v5: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:14:53.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:52 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:14:53.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:52 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:14:53.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:52 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:14:53.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:52 smithi062 ceph-mon[29666]: pgmap v5: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:14:53.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:52 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:14:53.414 DEBUG:teuthology.orchestra.run.smithi062:alertmanager.a> sudo journalctl -f -n 0 -u ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@alertmanager.a.service 2024-03-24T11:14:53.417 INFO:tasks.cephadm:Adding grafana.a on smithi139 2024-03-24T11:14:53.418 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph orch apply grafana '1;smithi139=a' 2024-03-24T11:14:54.005 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:53 smithi139 ceph-mon[25719]: from='client.24386 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi062=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:14:54.006 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:53 smithi139 ceph-mon[25719]: Saving service alertmanager spec with placement smithi062=a;count:1 2024-03-24T11:14:54.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:53 smithi062 ceph-mon[24799]: from='client.24386 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi062=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:14:54.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:53 smithi062 ceph-mon[24799]: Saving service alertmanager spec with placement smithi062=a;count:1 2024-03-24T11:14:54.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:53 smithi062 ceph-mon[29666]: from='client.24386 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi062=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:14:54.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:53 smithi062 ceph-mon[29666]: Saving service alertmanager spec with placement smithi062=a;count:1 2024-03-24T11:14:55.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:54 smithi139 ceph-mon[25719]: pgmap v6: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:14:55.128 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:54 smithi062 ceph-mon[24799]: pgmap v6: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:14:55.129 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:54 smithi062 ceph-mon[29666]: pgmap v6: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:14:56.781 INFO:teuthology.orchestra.run.smithi139.stdout:Scheduled grafana update... 2024-03-24T11:14:57.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:56 smithi139 ceph-mon[25719]: pgmap v7: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:14:57.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:56 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:14:57.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:56 smithi062 ceph-mon[24799]: pgmap v7: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:14:57.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:56 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:14:57.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:56 smithi062 ceph-mon[29666]: pgmap v7: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:14:57.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:56 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:14:57.960 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:57 smithi062 ceph-mon[24799]: from='client.24392 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi139=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:14:57.960 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:57 smithi062 ceph-mon[24799]: Saving service grafana spec with placement smithi139=a;count:1 2024-03-24T11:14:57.960 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:57 smithi062 ceph-mon[29666]: from='client.24392 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi139=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:14:57.960 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:57 smithi062 ceph-mon[29666]: Saving service grafana spec with placement smithi139=a;count:1 2024-03-24T11:14:58.243 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:57 smithi139 ceph-mon[25719]: from='client.24392 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi139=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:14:58.243 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:57 smithi139 ceph-mon[25719]: Saving service grafana spec with placement smithi139=a;count:1 2024-03-24T11:14:58.280 DEBUG:teuthology.orchestra.run.smithi139:grafana.a> sudo journalctl -f -n 0 -u ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@grafana.a.service 2024-03-24T11:14:58.283 INFO:tasks.cephadm:Setting up client nodes... 2024-03-24T11:14:58.285 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-24T11:14:58.862 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:14:58 smithi062 ceph-mon[29666]: pgmap v8: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:14:59.175 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:14:58 smithi062 ceph-mon[24799]: pgmap v8: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:14:59.248 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:14:58 smithi139 ceph-mon[25719]: pgmap v8: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:00.321 INFO:teuthology.orchestra.run.smithi062.stdout:[client.0] 2024-03-24T11:15:00.321 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQA0CwBm2TzzEhAAnFCEUNVsCMJPmRo3bPyawQ== 2024-03-24T11:15:00.916 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:00 smithi062 ceph-mon[29666]: pgmap v9: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:00.916 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:00 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/1421022889' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-24T11:15:00.916 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:00 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/1421022889' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-24T11:15:00.916 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:00 smithi062 ceph-mon[24799]: pgmap v9: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:00.916 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:00 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/1421022889' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-24T11:15:00.916 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:00 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/1421022889' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-24T11:15:00.941 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-03-24T11:15:00.941 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-24T11:15:00.941 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-24T11:15:00.986 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-24T11:15:00.995 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:00 smithi139 ceph-mon[25719]: pgmap v9: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:00.995 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:00 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/1421022889' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-24T11:15:00.995 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:00 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/1421022889' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-24T11:15:03.121 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:02 smithi139 ceph-mon[25719]: pgmap v10: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:03.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:02 smithi062 ceph-mon[24799]: pgmap v10: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:03.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:02 smithi062 ceph-mon[29666]: pgmap v10: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:04.227 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:04 smithi062 systemd[1]: Starting Ceph node-exporter.a for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:15:04.582 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:04 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:04.582 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:04 smithi062 ceph-mon[24799]: Deploying daemon node-exporter.a on smithi062 2024-03-24T11:15:04.582 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:04 smithi062 ceph-mon[24799]: pgmap v11: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:04.583 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:04 smithi062 bash[45463]: Trying to pull quay.io/prometheus/node-exporter:v1.3.1... 2024-03-24T11:15:04.583 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:04 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:04.583 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:04 smithi062 ceph-mon[29666]: Deploying daemon node-exporter.a on smithi062 2024-03-24T11:15:04.583 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:04 smithi062 ceph-mon[29666]: pgmap v11: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:04.671 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:04 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:04.671 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:04 smithi139 ceph-mon[25719]: Deploying daemon node-exporter.a on smithi062 2024-03-24T11:15:04.671 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:04 smithi139 ceph-mon[25719]: pgmap v11: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:04.976 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:04 smithi062 bash[45463]: Getting image source signatures 2024-03-24T11:15:04.977 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:04 smithi062 bash[45463]: Copying blob sha256:aa2a8d90b84cb2a9c422e7005cd166a008ccf22ef5d7d4f07128478585ce35ea 2024-03-24T11:15:04.977 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:04 smithi062 bash[45463]: Copying blob sha256:b5db1e299295edf3005515ab7879c1df64a33c185d3a7a23aa4dcaa17d26f7b3 2024-03-24T11:15:04.977 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:04 smithi062 bash[45463]: Copying blob sha256:b45d31ee2d7f9f452678a85b0c837c29e12089f31ee8dbac6c8c24dfa4054a30 2024-03-24T11:15:06.155 INFO:teuthology.orchestra.run.smithi139.stdout:[client.1] 2024-03-24T11:15:06.155 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQA6CwBmWAnXCBAAf98Dqn2g/9XSzbpdlVOMTg== 2024-03-24T11:15:06.865 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-03-24T11:15:06.865 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-03-24T11:15:06.865 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-24T11:15:06.889 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:15:06 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.139 - - [24/Mar/2024:11:15:06] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:15:06.890 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:06 smithi062 bash[45463]: Copying config sha256:1dbe0e931976487e20e5cfb272087e08a9779c88fd5e9617ed7042dd9751ec26 2024-03-24T11:15:06.948 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-24T11:15:06.949 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-24T11:15:06.949 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph mgr dump --format=json 2024-03-24T11:15:07.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:06 smithi139 ceph-mon[25719]: pgmap v12: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:07.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:06 smithi139 ceph-mon[25719]: from='client.? 172.21.15.139:0/1231101717' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-24T11:15:07.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:06 smithi139 ceph-mon[25719]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-24T11:15:07.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:06 smithi139 ceph-mon[25719]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-24T11:15:07.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:06 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:07.173 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T11:15:07.175 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:06 smithi062 bash[45463]: Writing manifest to image destination 2024-03-24T11:15:07.175 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:06 smithi062 ceph-mon[29666]: pgmap v12: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:07.176 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:06 smithi062 ceph-mon[29666]: from='client.? 172.21.15.139:0/1231101717' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-24T11:15:07.176 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:06 smithi062 ceph-mon[29666]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-24T11:15:07.176 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:06 smithi062 ceph-mon[29666]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-24T11:15:07.176 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:06 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:07.176 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:06 smithi062 ceph-mon[24799]: pgmap v12: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:07.176 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:06 smithi062 ceph-mon[24799]: from='client.? 172.21.15.139:0/1231101717' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-24T11:15:07.176 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:06 smithi062 ceph-mon[24799]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-24T11:15:07.176 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:06 smithi062 ceph-mon[24799]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-24T11:15:07.176 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:06 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:07.474 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 podman[45463]: 2024-03-24 11:15:07.165989881 +0000 UTC m=+3.018018126 image pull 1dbe0e931976487e20e5cfb272087e08a9779c88fd5e9617ed7042dd9751ec26 quay.io/prometheus/node-exporter:v1.3.1 2024-03-24T11:15:07.474 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 podman[45463]: 2024-03-24 11:15:07.28101581 +0000 UTC m=+3.133044041 container create bb795f9b0512317ac3febed1bcf5ad84def4a905653211fb92c7e95b6e55fe07 (image=quay.io/prometheus/node-exporter:v1.3.1, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-03-24T11:15:07.726 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 podman[45463]: 2024-03-24 11:15:07.483885765 +0000 UTC m=+3.335913997 container init bb795f9b0512317ac3febed1bcf5ad84def4a905653211fb92c7e95b6e55fe07 (image=quay.io/prometheus/node-exporter:v1.3.1, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-03-24T11:15:07.727 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 podman[45463]: 2024-03-24 11:15:07.486481732 +0000 UTC m=+3.338509968 container start bb795f9b0512317ac3febed1bcf5ad84def4a905653211fb92c7e95b6e55fe07 (image=quay.io/prometheus/node-exporter:v1.3.1, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-03-24T11:15:07.727 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[45570]: ts=2024-03-24T11:15:07.490Z caller=node_exporter.go:182 level=info msg="Starting node_exporter" version="(version=1.3.1, branch=HEAD, revision=a2321e7b940ddcff26873612bccdf7cd4c42b6b6)" 2024-03-24T11:15:07.727 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[45570]: ts=2024-03-24T11:15:07.490Z caller=node_exporter.go:183 level=info msg="Build context" build_context="(go=go1.17.3, user=root@243aafa5525c, date=20211205-11:09:49)" 2024-03-24T11:15:07.727 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[45570]: ts=2024-03-24T11:15:07.490Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+)($|/) 2024-03-24T11:15:07.727 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[45570]: ts=2024-03-24T11:15:07.490Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-03-24T11:15:07.727 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[45570]: ts=2024-03-24T11:15:07.491Z caller=node_exporter.go:108 level=info msg="Enabled collectors" 2024-03-24T11:15:07.727 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[45570]: ts=2024-03-24T11:15:07.491Z caller=node_exporter.go:115 level=info collector=arp 2024-03-24T11:15:07.727 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[45570]: ts=2024-03-24T11:15:07.491Z caller=node_exporter.go:115 level=info collector=bcache 2024-03-24T11:15:07.727 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[45570]: ts=2024-03-24T11:15:07.491Z caller=node_exporter.go:115 level=info collector=bonding 2024-03-24T11:15:07.727 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[45570]: ts=2024-03-24T11:15:07.491Z caller=node_exporter.go:115 level=info collector=btrfs 2024-03-24T11:15:07.727 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[45570]: ts=2024-03-24T11:15:07.491Z caller=node_exporter.go:115 level=info collector=conntrack 2024-03-24T11:15:07.727 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[45570]: ts=2024-03-24T11:15:07.491Z caller=node_exporter.go:115 level=info collector=cpu 2024-03-24T11:15:07.727 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[45570]: ts=2024-03-24T11:15:07.491Z caller=node_exporter.go:115 level=info collector=cpufreq 2024-03-24T11:15:07.727 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[45570]: ts=2024-03-24T11:15:07.491Z caller=node_exporter.go:115 level=info collector=diskstats 2024-03-24T11:15:07.728 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[45570]: ts=2024-03-24T11:15:07.491Z caller=node_exporter.go:115 level=info collector=dmi 2024-03-24T11:15:07.728 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[45570]: ts=2024-03-24T11:15:07.491Z caller=node_exporter.go:115 level=info collector=edac 2024-03-24T11:15:07.728 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[45570]: ts=2024-03-24T11:15:07.491Z caller=node_exporter.go:115 level=info collector=entropy 2024-03-24T11:15:07.728 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[45570]: ts=2024-03-24T11:15:07.491Z caller=node_exporter.go:115 level=info collector=fibrechannel 2024-03-24T11:15:07.728 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[45570]: ts=2024-03-24T11:15:07.491Z caller=node_exporter.go:115 level=info collector=filefd 2024-03-24T11:15:07.728 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[45570]: ts=2024-03-24T11:15:07.491Z caller=node_exporter.go:115 level=info collector=filesystem 2024-03-24T11:15:07.728 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[45570]: ts=2024-03-24T11:15:07.491Z caller=node_exporter.go:115 level=info collector=hwmon 2024-03-24T11:15:07.728 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[45570]: ts=2024-03-24T11:15:07.491Z caller=node_exporter.go:115 level=info collector=infiniband 2024-03-24T11:15:07.728 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[45570]: ts=2024-03-24T11:15:07.491Z caller=node_exporter.go:115 level=info collector=ipvs 2024-03-24T11:15:07.729 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[45570]: ts=2024-03-24T11:15:07.491Z caller=node_exporter.go:115 level=info collector=loadavg 2024-03-24T11:15:07.729 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[45570]: ts=2024-03-24T11:15:07.491Z caller=node_exporter.go:115 level=info collector=mdadm 2024-03-24T11:15:07.729 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[45570]: ts=2024-03-24T11:15:07.491Z caller=node_exporter.go:115 level=info collector=meminfo 2024-03-24T11:15:07.729 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[45570]: ts=2024-03-24T11:15:07.491Z caller=node_exporter.go:115 level=info collector=netclass 2024-03-24T11:15:07.729 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[45570]: ts=2024-03-24T11:15:07.491Z caller=node_exporter.go:115 level=info collector=netdev 2024-03-24T11:15:07.729 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[45570]: ts=2024-03-24T11:15:07.491Z caller=node_exporter.go:115 level=info collector=netstat 2024-03-24T11:15:07.729 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[45570]: ts=2024-03-24T11:15:07.491Z caller=node_exporter.go:115 level=info collector=nfs 2024-03-24T11:15:07.729 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[45570]: ts=2024-03-24T11:15:07.491Z caller=node_exporter.go:115 level=info collector=nfsd 2024-03-24T11:15:07.729 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[45570]: ts=2024-03-24T11:15:07.491Z caller=node_exporter.go:115 level=info collector=nvme 2024-03-24T11:15:07.729 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[45570]: ts=2024-03-24T11:15:07.491Z caller=node_exporter.go:115 level=info collector=os 2024-03-24T11:15:07.729 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[45570]: ts=2024-03-24T11:15:07.491Z caller=node_exporter.go:115 level=info collector=powersupplyclass 2024-03-24T11:15:07.729 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[45570]: ts=2024-03-24T11:15:07.491Z caller=node_exporter.go:115 level=info collector=pressure 2024-03-24T11:15:07.729 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[45570]: ts=2024-03-24T11:15:07.491Z caller=node_exporter.go:115 level=info collector=rapl 2024-03-24T11:15:07.729 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[45570]: ts=2024-03-24T11:15:07.491Z caller=node_exporter.go:115 level=info collector=schedstat 2024-03-24T11:15:07.729 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[45570]: ts=2024-03-24T11:15:07.491Z caller=node_exporter.go:115 level=info collector=sockstat 2024-03-24T11:15:07.729 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[45570]: ts=2024-03-24T11:15:07.491Z caller=node_exporter.go:115 level=info collector=softnet 2024-03-24T11:15:07.729 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[45570]: ts=2024-03-24T11:15:07.491Z caller=node_exporter.go:115 level=info collector=stat 2024-03-24T11:15:07.730 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[45570]: ts=2024-03-24T11:15:07.491Z caller=node_exporter.go:115 level=info collector=tapestats 2024-03-24T11:15:07.730 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[45570]: ts=2024-03-24T11:15:07.491Z caller=node_exporter.go:115 level=info collector=textfile 2024-03-24T11:15:07.730 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[45570]: ts=2024-03-24T11:15:07.491Z caller=node_exporter.go:115 level=info collector=thermal_zone 2024-03-24T11:15:07.730 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[45570]: ts=2024-03-24T11:15:07.491Z caller=node_exporter.go:115 level=info collector=time 2024-03-24T11:15:07.730 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[45570]: ts=2024-03-24T11:15:07.491Z caller=node_exporter.go:115 level=info collector=udp_queues 2024-03-24T11:15:07.730 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[45570]: ts=2024-03-24T11:15:07.491Z caller=node_exporter.go:115 level=info collector=uname 2024-03-24T11:15:07.730 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[45570]: ts=2024-03-24T11:15:07.491Z caller=node_exporter.go:115 level=info collector=vmstat 2024-03-24T11:15:07.730 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[45570]: ts=2024-03-24T11:15:07.491Z caller=node_exporter.go:115 level=info collector=xfs 2024-03-24T11:15:07.730 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[45570]: ts=2024-03-24T11:15:07.491Z caller=node_exporter.go:115 level=info collector=zfs 2024-03-24T11:15:07.730 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[45570]: ts=2024-03-24T11:15:07.491Z caller=node_exporter.go:199 level=info msg="Listening on" address=:9100 2024-03-24T11:15:07.730 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[45570]: ts=2024-03-24T11:15:07.491Z caller=tls_config.go:195 level=info msg="TLS is disabled." http2=false 2024-03-24T11:15:07.730 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 bash[45463]: bb795f9b0512317ac3febed1bcf5ad84def4a905653211fb92c7e95b6e55fe07 2024-03-24T11:15:07.730 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:15:07 smithi062 systemd[1]: Started Ceph node-exporter.a for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:15:08.802 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:08 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:08.802 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:08 smithi139 ceph-mon[25719]: Deploying daemon node-exporter.b on smithi139 2024-03-24T11:15:08.802 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:08 smithi139 ceph-mon[25719]: pgmap v13: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:08.802 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:08 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:08.890 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:08 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:08.890 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:08 smithi062 ceph-mon[29666]: Deploying daemon node-exporter.b on smithi139 2024-03-24T11:15:08.890 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:08 smithi062 ceph-mon[29666]: pgmap v13: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:08.890 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:08 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:08.891 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:08 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:08.891 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:08 smithi062 ceph-mon[24799]: Deploying daemon node-exporter.b on smithi139 2024-03-24T11:15:08.891 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:08 smithi062 ceph-mon[24799]: pgmap v13: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:08.891 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:08 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:09.226 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:15:08 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:15:08] "GET /metrics HTTP/1.1" 200 191500 "" "Prometheus/2.33.4" 2024-03-24T11:15:09.383 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-24T11:15:09.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:09 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/2927260108' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-24T11:15:09.932 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:09 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/2927260108' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-24T11:15:09.932 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:09 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/2927260108' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-24T11:15:10.268 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":20,"active_gid":14430,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":1600445979},{"type":"v1","addr":"172.21.15.62:6801","nonce":1600445979}]},"active_addr":"172.21.15.62:6801/1600445979","active_change":"2024-03-24T11:14:46.881136+0000","active_mgr_features":4540138303579357183,"available":true,"standbys":[{"gid":24350,"name":"x","mgr_features":4540138303579357183,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/library/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"docker.io/arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/library/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"docker.io/arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.62:8443/","prometheus":"http://172.21.15.62:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":60,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.62:0","nonce":2398349393}]},{"addrvec":[{"type":"v2","addr":"172.21.15.62:0","nonce":555071628}]},{"addrvec":[{"type":"v2","addr":"172.21.15.62:0","nonce":3932249987}]},{"addrvec":[{"type":"v2","addr":"172.21.15.62:0","nonce":793412542}]}]}} 2024-03-24T11:15:10.271 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-24T11:15:10.272 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-24T11:15:10.272 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-24T11:15:10.502 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T11:15:10.609 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:15:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.139 - - [24/Mar/2024:11:15:10] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:15:10.911 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:10 smithi062 ceph-mon[24799]: pgmap v14: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:10.911 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:10 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:10.911 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:10 smithi062 ceph-mon[29666]: pgmap v14: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:10.911 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:10 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:10.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:10 smithi139 ceph-mon[25719]: pgmap v14: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:10.999 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:10 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:10.999 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:15:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:15:10] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-03-24T11:15:10.999 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:10 smithi139 systemd[1]: Starting Ceph node-exporter.b for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:15:11.322 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:10 smithi139 bash[42630]: Trying to pull quay.io/prometheus/node-exporter:v1.3.1... 2024-03-24T11:15:11.607 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:11 smithi139 bash[42630]: Getting image source signatures 2024-03-24T11:15:11.608 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:11 smithi139 bash[42630]: Copying blob sha256:b5db1e299295edf3005515ab7879c1df64a33c185d3a7a23aa4dcaa17d26f7b3 2024-03-24T11:15:11.608 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:11 smithi139 bash[42630]: Copying blob sha256:b45d31ee2d7f9f452678a85b0c837c29e12089f31ee8dbac6c8c24dfa4054a30 2024-03-24T11:15:11.608 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:11 smithi139 bash[42630]: Copying blob sha256:aa2a8d90b84cb2a9c422e7005cd166a008ccf22ef5d7d4f07128478585ce35ea 2024-03-24T11:15:11.861 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:15:11 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.139 - - [24/Mar/2024:11:15:11] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:15:11.862 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:11 smithi062 ceph-mon[24799]: Detected new or changed devices on smithi139 2024-03-24T11:15:11.862 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:11 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:11.862 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:11 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:11.862 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:11 smithi062 ceph-mon[29666]: Detected new or changed devices on smithi139 2024-03-24T11:15:11.862 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:11 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:11.862 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:11 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:12.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:11 smithi139 ceph-mon[25719]: Detected new or changed devices on smithi139 2024-03-24T11:15:12.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:11 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:12.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:11 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:12.222 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:15:11 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.62 - - [24/Mar/2024:11:15:11] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:15:12.265 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-24T11:15:12.265 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":60,"fsid":"ebb89568-e9ce-11ee-95cd-87774f69a715","created":"2024-03-24T11:09:33.575451+0000","modified":"2024-03-24T11:14:46.880762+0000","last_up_change":"2024-03-24T11:14:27.827075+0000","last_in_change":"2024-03-24T11:14:08.484969+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-24T11:12:32.117868+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"83103b18-933d-437b-9762-583f85bc1776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1287363541},{"type":"v1","addr":"172.21.15.62:6803","nonce":1287363541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":1287363541},{"type":"v1","addr":"172.21.15.62:6805","nonce":1287363541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":1287363541},{"type":"v1","addr":"172.21.15.62:6809","nonce":1287363541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1287363541},{"type":"v1","addr":"172.21.15.62:6807","nonce":1287363541}]},"public_addr":"172.21.15.62:6803/1287363541","cluster_addr":"172.21.15.62:6805/1287363541","heartbeat_back_addr":"172.21.15.62:6809/1287363541","heartbeat_front_addr":"172.21.15.62:6807/1287363541","state":["exists","up"]},{"osd":1,"uuid":"515c2c9b-7bc4-4b00-9be7-8773cf551306","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1156951329},{"type":"v1","addr":"172.21.15.62:6811","nonce":1156951329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1156951329},{"type":"v1","addr":"172.21.15.62:6813","nonce":1156951329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1156951329},{"type":"v1","addr":"172.21.15.62:6817","nonce":1156951329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1156951329},{"type":"v1","addr":"172.21.15.62:6815","nonce":1156951329}]},"public_addr":"172.21.15.62:6811/1156951329","cluster_addr":"172.21.15.62:6813/1156951329","heartbeat_back_addr":"172.21.15.62:6817/1156951329","heartbeat_front_addr":"172.21.15.62:6815/1156951329","state":["exists","up"]},{"osd":2,"uuid":"569ff950-c8b0-48fc-b137-4598571ff36c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2474659851},{"type":"v1","addr":"172.21.15.62:6819","nonce":2474659851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2474659851},{"type":"v1","addr":"172.21.15.62:6821","nonce":2474659851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":2474659851},{"type":"v1","addr":"172.21.15.62:6825","nonce":2474659851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2474659851},{"type":"v1","addr":"172.21.15.62:6823","nonce":2474659851}]},"public_addr":"172.21.15.62:6819/2474659851","cluster_addr":"172.21.15.62:6821/2474659851","heartbeat_back_addr":"172.21.15.62:6825/2474659851","heartbeat_front_addr":"172.21.15.62:6823/2474659851","state":["exists","up"]},{"osd":3,"uuid":"e0c09510-dcba-4332-9c02-3842c992612b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":2104856662},{"type":"v1","addr":"172.21.15.62:6827","nonce":2104856662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":2104856662},{"type":"v1","addr":"172.21.15.62:6829","nonce":2104856662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2104856662},{"type":"v1","addr":"172.21.15.62:6833","nonce":2104856662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":2104856662},{"type":"v1","addr":"172.21.15.62:6831","nonce":2104856662}]},"public_addr":"172.21.15.62:6827/2104856662","cluster_addr":"172.21.15.62:6829/2104856662","heartbeat_back_addr":"172.21.15.62:6833/2104856662","heartbeat_front_addr":"172.21.15.62:6831/2104856662","state":["exists","up"]},{"osd":4,"uuid":"8c96358d-d11f-4f66-a433-3e4dc3024322","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3866478379},{"type":"v1","addr":"172.21.15.139:6801","nonce":3866478379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3866478379},{"type":"v1","addr":"172.21.15.139:6803","nonce":3866478379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3866478379},{"type":"v1","addr":"172.21.15.139:6807","nonce":3866478379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3866478379},{"type":"v1","addr":"172.21.15.139:6805","nonce":3866478379}]},"public_addr":"172.21.15.139:6801/3866478379","cluster_addr":"172.21.15.139:6803/3866478379","heartbeat_back_addr":"172.21.15.139:6807/3866478379","heartbeat_front_addr":"172.21.15.139:6805/3866478379","state":["exists","up"]},{"osd":5,"uuid":"adf8b0ea-4d03-4bc4-bca4-2b6aad9e967d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1384318027},{"type":"v1","addr":"172.21.15.139:6809","nonce":1384318027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1384318027},{"type":"v1","addr":"172.21.15.139:6811","nonce":1384318027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1384318027},{"type":"v1","addr":"172.21.15.139:6815","nonce":1384318027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1384318027},{"type":"v1","addr":"172.21.15.139:6813","nonce":1384318027}]},"public_addr":"172.21.15.139:6809/1384318027","cluster_addr":"172.21.15.139:6811/1384318027","heartbeat_back_addr":"172.21.15.139:6815/1384318027","heartbeat_front_addr":"172.21.15.139:6813/1384318027","state":["exists","up"]},{"osd":6,"uuid":"f9790a56-fa0c-4908-8fbc-c347ecefa2df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":2768426846},{"type":"v1","addr":"172.21.15.139:6817","nonce":2768426846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":2768426846},{"type":"v1","addr":"172.21.15.139:6819","nonce":2768426846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":2768426846},{"type":"v1","addr":"172.21.15.139:6823","nonce":2768426846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":2768426846},{"type":"v1","addr":"172.21.15.139:6821","nonce":2768426846}]},"public_addr":"172.21.15.139:6817/2768426846","cluster_addr":"172.21.15.139:6819/2768426846","heartbeat_back_addr":"172.21.15.139:6823/2768426846","heartbeat_front_addr":"172.21.15.139:6821/2768426846","state":["exists","up"]},{"osd":7,"uuid":"5422547c-7038-4654-807c-e118f84575f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2297684619},{"type":"v1","addr":"172.21.15.139:6825","nonce":2297684619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2297684619},{"type":"v1","addr":"172.21.15.139:6827","nonce":2297684619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2297684619},{"type":"v1","addr":"172.21.15.139:6831","nonce":2297684619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2297684619},{"type":"v1","addr":"172.21.15.139:6829","nonce":2297684619}]},"public_addr":"172.21.15.139:6825/2297684619","cluster_addr":"172.21.15.139:6827/2297684619","heartbeat_back_addr":"172.21.15.139:6831/2297684619","heartbeat_front_addr":"172.21.15.139:6829/2297684619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T11:11:44.756685+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T11:12:06.837947+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T11:12:28.819088+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T11:12:52.203208+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T11:13:13.838606+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T11:13:36.993722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T11:14:00.888906+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T11:14:26.280662+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/621428596":"2024-03-25T11:14:46.880732+0000","172.21.15.62:6800/621428596":"2024-03-25T11:14:46.880732+0000","172.21.15.62:0/1654711111":"2024-03-25T11:14:46.880732+0000","172.21.15.62:6801/2609891207":"2024-03-25T11:10:23.129060+0000","172.21.15.62:0/2764833578":"2024-03-25T11:10:23.129060+0000","172.21.15.62:6800/2609891207":"2024-03-25T11:10:23.129060+0000","172.21.15.62:0/81242112":"2024-03-25T11:10:23.129060+0000","172.21.15.62:0/2403755398":"2024-03-25T11:14:46.880732+0000","172.21.15.62:0/2039307015":"2024-03-25T11:10:00.759801+0000","172.21.15.62:0/4080145397":"2024-03-25T11:10:23.129060+0000","172.21.15.62:0/936493210":"2024-03-25T11:10:00.759801+0000","172.21.15.62:0/2103795741":"2024-03-25T11:14:46.880732+0000","172.21.15.62:0/106493134":"2024-03-25T11:10:00.759801+0000","172.21.15.62:6800/1616593108":"2024-03-25T11:10:00.759801+0000","172.21.15.62:0/2488622750":"2024-03-25T11:14:46.880732+0000","172.21.15.62:6801/1616593108":"2024-03-25T11:10:00.759801+0000"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-24T11:15:13.026 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:12 smithi139 bash[42630]: Copying config sha256:1dbe0e931976487e20e5cfb272087e08a9779c88fd5e9617ed7042dd9751ec26 2024-03-24T11:15:13.027 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:12 smithi139 bash[42630]: Writing manifest to image destination 2024-03-24T11:15:13.027 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:12 smithi139 ceph-mon[25719]: pgmap v15: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:13.027 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:12 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:13.027 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:12 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:13.027 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:12 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/837315395' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T11:15:13.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:12 smithi062 ceph-mon[24799]: pgmap v15: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:13.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:12 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:13.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:12 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:13.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:12 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/837315395' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T11:15:13.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:12 smithi062 ceph-mon[29666]: pgmap v15: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:13.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:12 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:13.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:12 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:13.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:12 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/837315395' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T11:15:13.283 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 podman[42630]: 2024-03-24 11:15:12.983140509 +0000 UTC m=+2.079505781 image pull 1dbe0e931976487e20e5cfb272087e08a9779c88fd5e9617ed7042dd9751ec26 quay.io/prometheus/node-exporter:v1.3.1 2024-03-24T11:15:13.283 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 podman[42630]: 2024-03-24 11:15:13.090096174 +0000 UTC m=+2.186461431 container create 8921c8c780f7c30d7013b943c506bc0719456cf780f3d33a796da7e4450c05f8 (image=quay.io/prometheus/node-exporter:v1.3.1, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-03-24T11:15:13.354 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-24T11:15:13.354 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-24T11:15:13.608 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 podman[42630]: 2024-03-24 11:15:13.283928117 +0000 UTC m=+2.380293384 container init 8921c8c780f7c30d7013b943c506bc0719456cf780f3d33a796da7e4450c05f8 (image=quay.io/prometheus/node-exporter:v1.3.1, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-03-24T11:15:13.608 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 podman[42630]: 2024-03-24 11:15:13.286827299 +0000 UTC m=+2.383192565 container start 8921c8c780f7c30d7013b943c506bc0719456cf780f3d33a796da7e4450c05f8 (image=quay.io/prometheus/node-exporter:v1.3.1, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-03-24T11:15:13.609 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[42751]: ts=2024-03-24T11:15:13.290Z caller=node_exporter.go:182 level=info msg="Starting node_exporter" version="(version=1.3.1, branch=HEAD, revision=a2321e7b940ddcff26873612bccdf7cd4c42b6b6)" 2024-03-24T11:15:13.609 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[42751]: ts=2024-03-24T11:15:13.290Z caller=node_exporter.go:183 level=info msg="Build context" build_context="(go=go1.17.3, user=root@243aafa5525c, date=20211205-11:09:49)" 2024-03-24T11:15:13.609 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[42751]: ts=2024-03-24T11:15:13.290Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+)($|/) 2024-03-24T11:15:13.609 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[42751]: ts=2024-03-24T11:15:13.290Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-03-24T11:15:13.609 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[42751]: ts=2024-03-24T11:15:13.291Z caller=node_exporter.go:108 level=info msg="Enabled collectors" 2024-03-24T11:15:13.609 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[42751]: ts=2024-03-24T11:15:13.291Z caller=node_exporter.go:115 level=info collector=arp 2024-03-24T11:15:13.609 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[42751]: ts=2024-03-24T11:15:13.291Z caller=node_exporter.go:115 level=info collector=bcache 2024-03-24T11:15:13.609 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[42751]: ts=2024-03-24T11:15:13.291Z caller=node_exporter.go:115 level=info collector=bonding 2024-03-24T11:15:13.609 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[42751]: ts=2024-03-24T11:15:13.291Z caller=node_exporter.go:115 level=info collector=btrfs 2024-03-24T11:15:13.609 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[42751]: ts=2024-03-24T11:15:13.291Z caller=node_exporter.go:115 level=info collector=conntrack 2024-03-24T11:15:13.609 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[42751]: ts=2024-03-24T11:15:13.291Z caller=node_exporter.go:115 level=info collector=cpu 2024-03-24T11:15:13.609 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[42751]: ts=2024-03-24T11:15:13.291Z caller=node_exporter.go:115 level=info collector=cpufreq 2024-03-24T11:15:13.609 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[42751]: ts=2024-03-24T11:15:13.291Z caller=node_exporter.go:115 level=info collector=diskstats 2024-03-24T11:15:13.609 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[42751]: ts=2024-03-24T11:15:13.291Z caller=node_exporter.go:115 level=info collector=dmi 2024-03-24T11:15:13.609 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[42751]: ts=2024-03-24T11:15:13.291Z caller=node_exporter.go:115 level=info collector=edac 2024-03-24T11:15:13.610 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[42751]: ts=2024-03-24T11:15:13.291Z caller=node_exporter.go:115 level=info collector=entropy 2024-03-24T11:15:13.610 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[42751]: ts=2024-03-24T11:15:13.291Z caller=node_exporter.go:115 level=info collector=fibrechannel 2024-03-24T11:15:13.610 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[42751]: ts=2024-03-24T11:15:13.291Z caller=node_exporter.go:115 level=info collector=filefd 2024-03-24T11:15:13.610 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[42751]: ts=2024-03-24T11:15:13.291Z caller=node_exporter.go:115 level=info collector=filesystem 2024-03-24T11:15:13.610 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[42751]: ts=2024-03-24T11:15:13.291Z caller=node_exporter.go:115 level=info collector=hwmon 2024-03-24T11:15:13.610 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[42751]: ts=2024-03-24T11:15:13.291Z caller=node_exporter.go:115 level=info collector=infiniband 2024-03-24T11:15:13.610 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[42751]: ts=2024-03-24T11:15:13.291Z caller=node_exporter.go:115 level=info collector=ipvs 2024-03-24T11:15:13.610 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[42751]: ts=2024-03-24T11:15:13.291Z caller=node_exporter.go:115 level=info collector=loadavg 2024-03-24T11:15:13.611 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[42751]: ts=2024-03-24T11:15:13.291Z caller=node_exporter.go:115 level=info collector=mdadm 2024-03-24T11:15:13.611 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[42751]: ts=2024-03-24T11:15:13.291Z caller=node_exporter.go:115 level=info collector=meminfo 2024-03-24T11:15:13.611 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[42751]: ts=2024-03-24T11:15:13.291Z caller=node_exporter.go:115 level=info collector=netclass 2024-03-24T11:15:13.611 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[42751]: ts=2024-03-24T11:15:13.291Z caller=node_exporter.go:115 level=info collector=netdev 2024-03-24T11:15:13.611 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[42751]: ts=2024-03-24T11:15:13.291Z caller=node_exporter.go:115 level=info collector=netstat 2024-03-24T11:15:13.611 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[42751]: ts=2024-03-24T11:15:13.291Z caller=node_exporter.go:115 level=info collector=nfs 2024-03-24T11:15:13.611 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[42751]: ts=2024-03-24T11:15:13.291Z caller=node_exporter.go:115 level=info collector=nfsd 2024-03-24T11:15:13.611 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[42751]: ts=2024-03-24T11:15:13.291Z caller=node_exporter.go:115 level=info collector=nvme 2024-03-24T11:15:13.611 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[42751]: ts=2024-03-24T11:15:13.291Z caller=node_exporter.go:115 level=info collector=os 2024-03-24T11:15:13.611 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[42751]: ts=2024-03-24T11:15:13.291Z caller=node_exporter.go:115 level=info collector=powersupplyclass 2024-03-24T11:15:13.611 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[42751]: ts=2024-03-24T11:15:13.291Z caller=node_exporter.go:115 level=info collector=pressure 2024-03-24T11:15:13.611 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[42751]: ts=2024-03-24T11:15:13.291Z caller=node_exporter.go:115 level=info collector=rapl 2024-03-24T11:15:13.611 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[42751]: ts=2024-03-24T11:15:13.291Z caller=node_exporter.go:115 level=info collector=schedstat 2024-03-24T11:15:13.611 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[42751]: ts=2024-03-24T11:15:13.291Z caller=node_exporter.go:115 level=info collector=sockstat 2024-03-24T11:15:13.611 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[42751]: ts=2024-03-24T11:15:13.291Z caller=node_exporter.go:115 level=info collector=softnet 2024-03-24T11:15:13.612 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[42751]: ts=2024-03-24T11:15:13.291Z caller=node_exporter.go:115 level=info collector=stat 2024-03-24T11:15:13.612 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[42751]: ts=2024-03-24T11:15:13.291Z caller=node_exporter.go:115 level=info collector=tapestats 2024-03-24T11:15:13.612 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[42751]: ts=2024-03-24T11:15:13.291Z caller=node_exporter.go:115 level=info collector=textfile 2024-03-24T11:15:13.612 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[42751]: ts=2024-03-24T11:15:13.291Z caller=node_exporter.go:115 level=info collector=thermal_zone 2024-03-24T11:15:13.612 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[42751]: ts=2024-03-24T11:15:13.291Z caller=node_exporter.go:115 level=info collector=time 2024-03-24T11:15:13.612 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[42751]: ts=2024-03-24T11:15:13.291Z caller=node_exporter.go:115 level=info collector=udp_queues 2024-03-24T11:15:13.612 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[42751]: ts=2024-03-24T11:15:13.291Z caller=node_exporter.go:115 level=info collector=uname 2024-03-24T11:15:13.612 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[42751]: ts=2024-03-24T11:15:13.291Z caller=node_exporter.go:115 level=info collector=vmstat 2024-03-24T11:15:13.612 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[42751]: ts=2024-03-24T11:15:13.291Z caller=node_exporter.go:115 level=info collector=xfs 2024-03-24T11:15:13.612 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[42751]: ts=2024-03-24T11:15:13.291Z caller=node_exporter.go:115 level=info collector=zfs 2024-03-24T11:15:13.612 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[42751]: ts=2024-03-24T11:15:13.291Z caller=node_exporter.go:199 level=info msg="Listening on" address=:9100 2024-03-24T11:15:13.612 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[42751]: ts=2024-03-24T11:15:13.291Z caller=tls_config.go:195 level=info msg="TLS is disabled." http2=false 2024-03-24T11:15:13.612 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 bash[42630]: 8921c8c780f7c30d7013b943c506bc0719456cf780f3d33a796da7e4450c05f8 2024-03-24T11:15:13.612 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:15:13 smithi139 systemd[1]: Started Ceph node-exporter.b for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:15:13.613 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T11:15:14.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:14 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:14.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:14 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:15:14.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:14 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:15:14.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:14 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:15:14.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:14 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:14.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:14 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:15:14.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:14 smithi062 ceph-mon[24799]: Reconfiguring prometheus.a (dependencies changed)... 2024-03-24T11:15:14.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:14 smithi062 ceph-mon[24799]: Reconfiguring daemon prometheus.a on smithi139 2024-03-24T11:15:14.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:14 smithi062 ceph-mon[24799]: pgmap v16: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:14.717 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:14 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:14.717 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:14 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:15:14.717 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:14 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:15:14.717 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:14 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:15:14.717 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:14 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:14.717 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:14 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:15:14.717 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:14 smithi062 ceph-mon[29666]: Reconfiguring prometheus.a (dependencies changed)... 2024-03-24T11:15:14.717 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:14 smithi062 ceph-mon[29666]: Reconfiguring daemon prometheus.a on smithi139 2024-03-24T11:15:14.717 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:14 smithi062 ceph-mon[29666]: pgmap v16: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:14.722 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:14 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:14.722 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:14 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:15:14.722 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:14 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:15:14.722 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:14 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:15:14.722 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:14 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:14.722 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:14 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:15:14.722 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:14 smithi139 ceph-mon[25719]: Reconfiguring prometheus.a (dependencies changed)... 2024-03-24T11:15:14.722 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:14 smithi139 ceph-mon[25719]: Reconfiguring daemon prometheus.a on smithi139 2024-03-24T11:15:14.722 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:14 smithi139 ceph-mon[25719]: pgmap v16: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:16.954 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:16 smithi139 ceph-mon[25719]: pgmap v17: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:17.185 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:16 smithi062 ceph-mon[24799]: pgmap v17: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:17.186 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:16 smithi062 ceph-mon[29666]: pgmap v17: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:17.477 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-24T11:15:17.477 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":60,"fsid":"ebb89568-e9ce-11ee-95cd-87774f69a715","created":"2024-03-24T11:09:33.575451+0000","modified":"2024-03-24T11:14:46.880762+0000","last_up_change":"2024-03-24T11:14:27.827075+0000","last_in_change":"2024-03-24T11:14:08.484969+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-24T11:12:32.117868+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"83103b18-933d-437b-9762-583f85bc1776","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1287363541},{"type":"v1","addr":"172.21.15.62:6803","nonce":1287363541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":1287363541},{"type":"v1","addr":"172.21.15.62:6805","nonce":1287363541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":1287363541},{"type":"v1","addr":"172.21.15.62:6809","nonce":1287363541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1287363541},{"type":"v1","addr":"172.21.15.62:6807","nonce":1287363541}]},"public_addr":"172.21.15.62:6803/1287363541","cluster_addr":"172.21.15.62:6805/1287363541","heartbeat_back_addr":"172.21.15.62:6809/1287363541","heartbeat_front_addr":"172.21.15.62:6807/1287363541","state":["exists","up"]},{"osd":1,"uuid":"515c2c9b-7bc4-4b00-9be7-8773cf551306","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1156951329},{"type":"v1","addr":"172.21.15.62:6811","nonce":1156951329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1156951329},{"type":"v1","addr":"172.21.15.62:6813","nonce":1156951329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1156951329},{"type":"v1","addr":"172.21.15.62:6817","nonce":1156951329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1156951329},{"type":"v1","addr":"172.21.15.62:6815","nonce":1156951329}]},"public_addr":"172.21.15.62:6811/1156951329","cluster_addr":"172.21.15.62:6813/1156951329","heartbeat_back_addr":"172.21.15.62:6817/1156951329","heartbeat_front_addr":"172.21.15.62:6815/1156951329","state":["exists","up"]},{"osd":2,"uuid":"569ff950-c8b0-48fc-b137-4598571ff36c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2474659851},{"type":"v1","addr":"172.21.15.62:6819","nonce":2474659851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2474659851},{"type":"v1","addr":"172.21.15.62:6821","nonce":2474659851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":2474659851},{"type":"v1","addr":"172.21.15.62:6825","nonce":2474659851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2474659851},{"type":"v1","addr":"172.21.15.62:6823","nonce":2474659851}]},"public_addr":"172.21.15.62:6819/2474659851","cluster_addr":"172.21.15.62:6821/2474659851","heartbeat_back_addr":"172.21.15.62:6825/2474659851","heartbeat_front_addr":"172.21.15.62:6823/2474659851","state":["exists","up"]},{"osd":3,"uuid":"e0c09510-dcba-4332-9c02-3842c992612b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":2104856662},{"type":"v1","addr":"172.21.15.62:6827","nonce":2104856662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":2104856662},{"type":"v1","addr":"172.21.15.62:6829","nonce":2104856662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2104856662},{"type":"v1","addr":"172.21.15.62:6833","nonce":2104856662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":2104856662},{"type":"v1","addr":"172.21.15.62:6831","nonce":2104856662}]},"public_addr":"172.21.15.62:6827/2104856662","cluster_addr":"172.21.15.62:6829/2104856662","heartbeat_back_addr":"172.21.15.62:6833/2104856662","heartbeat_front_addr":"172.21.15.62:6831/2104856662","state":["exists","up"]},{"osd":4,"uuid":"8c96358d-d11f-4f66-a433-3e4dc3024322","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":3866478379},{"type":"v1","addr":"172.21.15.139:6801","nonce":3866478379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":3866478379},{"type":"v1","addr":"172.21.15.139:6803","nonce":3866478379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":3866478379},{"type":"v1","addr":"172.21.15.139:6807","nonce":3866478379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":3866478379},{"type":"v1","addr":"172.21.15.139:6805","nonce":3866478379}]},"public_addr":"172.21.15.139:6801/3866478379","cluster_addr":"172.21.15.139:6803/3866478379","heartbeat_back_addr":"172.21.15.139:6807/3866478379","heartbeat_front_addr":"172.21.15.139:6805/3866478379","state":["exists","up"]},{"osd":5,"uuid":"adf8b0ea-4d03-4bc4-bca4-2b6aad9e967d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":1384318027},{"type":"v1","addr":"172.21.15.139:6809","nonce":1384318027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":1384318027},{"type":"v1","addr":"172.21.15.139:6811","nonce":1384318027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":1384318027},{"type":"v1","addr":"172.21.15.139:6815","nonce":1384318027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":1384318027},{"type":"v1","addr":"172.21.15.139:6813","nonce":1384318027}]},"public_addr":"172.21.15.139:6809/1384318027","cluster_addr":"172.21.15.139:6811/1384318027","heartbeat_back_addr":"172.21.15.139:6815/1384318027","heartbeat_front_addr":"172.21.15.139:6813/1384318027","state":["exists","up"]},{"osd":6,"uuid":"f9790a56-fa0c-4908-8fbc-c347ecefa2df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":2768426846},{"type":"v1","addr":"172.21.15.139:6817","nonce":2768426846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":2768426846},{"type":"v1","addr":"172.21.15.139:6819","nonce":2768426846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":2768426846},{"type":"v1","addr":"172.21.15.139:6823","nonce":2768426846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":2768426846},{"type":"v1","addr":"172.21.15.139:6821","nonce":2768426846}]},"public_addr":"172.21.15.139:6817/2768426846","cluster_addr":"172.21.15.139:6819/2768426846","heartbeat_back_addr":"172.21.15.139:6823/2768426846","heartbeat_front_addr":"172.21.15.139:6821/2768426846","state":["exists","up"]},{"osd":7,"uuid":"5422547c-7038-4654-807c-e118f84575f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2297684619},{"type":"v1","addr":"172.21.15.139:6825","nonce":2297684619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":2297684619},{"type":"v1","addr":"172.21.15.139:6827","nonce":2297684619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":2297684619},{"type":"v1","addr":"172.21.15.139:6831","nonce":2297684619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":2297684619},{"type":"v1","addr":"172.21.15.139:6829","nonce":2297684619}]},"public_addr":"172.21.15.139:6825/2297684619","cluster_addr":"172.21.15.139:6827/2297684619","heartbeat_back_addr":"172.21.15.139:6831/2297684619","heartbeat_front_addr":"172.21.15.139:6829/2297684619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T11:11:44.756685+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T11:12:06.837947+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T11:12:28.819088+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T11:12:52.203208+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T11:13:13.838606+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T11:13:36.993722+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T11:14:00.888906+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138303579357183,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T11:14:26.280662+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/621428596":"2024-03-25T11:14:46.880732+0000","172.21.15.62:6800/621428596":"2024-03-25T11:14:46.880732+0000","172.21.15.62:0/1654711111":"2024-03-25T11:14:46.880732+0000","172.21.15.62:6801/2609891207":"2024-03-25T11:10:23.129060+0000","172.21.15.62:0/2764833578":"2024-03-25T11:10:23.129060+0000","172.21.15.62:6800/2609891207":"2024-03-25T11:10:23.129060+0000","172.21.15.62:0/81242112":"2024-03-25T11:10:23.129060+0000","172.21.15.62:0/2403755398":"2024-03-25T11:14:46.880732+0000","172.21.15.62:0/2039307015":"2024-03-25T11:10:00.759801+0000","172.21.15.62:0/4080145397":"2024-03-25T11:10:23.129060+0000","172.21.15.62:0/936493210":"2024-03-25T11:10:00.759801+0000","172.21.15.62:0/2103795741":"2024-03-25T11:14:46.880732+0000","172.21.15.62:0/106493134":"2024-03-25T11:10:00.759801+0000","172.21.15.62:6800/1616593108":"2024-03-25T11:10:00.759801+0000","172.21.15.62:0/2488622750":"2024-03-25T11:14:46.880732+0000","172.21.15.62:6801/1616593108":"2024-03-25T11:10:00.759801+0000"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-24T11:15:17.798 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:15:17 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.62 - - [24/Mar/2024:11:15:17] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:15:17.858 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:17 smithi139 podman[43036]: 2024-03-24 11:15:17.604207183 +0000 UTC m=+0.137620095 container died 63a9d7160159aeec2f29c0ee353054061735472fbc70a82b2061b9e296b3ed50 (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-24T11:15:18.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:17 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/3897414771' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T11:15:18.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:17 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:18.216 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:17 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/3897414771' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T11:15:18.216 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:17 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:18.236 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:17 smithi139 podman[43036]: 2024-03-24 11:15:17.977143723 +0000 UTC m=+0.510556641 container cleanup 63a9d7160159aeec2f29c0ee353054061735472fbc70a82b2061b9e296b3ed50 (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-24T11:15:18.236 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:17 smithi139 bash[43036]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a 2024-03-24T11:15:18.236 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:17 smithi139 bash[43083]: Error: no container with name or ID "ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus.a" found: no such container 2024-03-24T11:15:18.236 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:17 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/3897414771' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T11:15:18.236 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:17 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:18.240 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-24T11:15:18.240 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-24T11:15:18.241 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-24T11:15:18.241 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-24T11:15:18.242 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-24T11:15:18.243 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-24T11:15:18.243 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-03-24T11:15:18.244 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-03-24T11:15:18.577 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:18 smithi139 podman[43060]: 2024-03-24 11:15:18.236330128 +0000 UTC m=+0.634505086 container remove 63a9d7160159aeec2f29c0ee353054061735472fbc70a82b2061b9e296b3ed50 (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-24T11:15:18.577 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:18 smithi139 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@prometheus.a.service: Deactivated successfully. 2024-03-24T11:15:18.577 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:18 smithi139 systemd[1]: Stopped Ceph prometheus.a for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:15:18.577 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:18 smithi139 systemd[1]: Starting Ceph prometheus.a for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:15:18.577 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:18 smithi139 podman[43155]: 2024-03-24 11:15:18.462233936 +0000 UTC m=+0.114320964 container create 29c5ca3f2ecc25784888c73321b74bcbdc5aed758f5937467948bd71a1497142 (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-24T11:15:18.577 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:18 smithi139 podman[43155]: 2024-03-24 11:15:18.369562458 +0000 UTC m=+0.021649507 image pull 514e6a882f6e74806a5856468489eeff8d7106095557578da96935e4d0ba4d9d quay.io/prometheus/prometheus:v2.33.4 2024-03-24T11:15:18.776 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T11:15:18.810 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T11:15:18.840 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T11:15:18.844 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T11:15:18.845 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T11:15:19.049 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:18 smithi139 ceph-mon[25719]: pgmap v18: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:19.049 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:18 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:19.049 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:18 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:19.049 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:18 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T11:15:19.049 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:18 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://172.21.15.139:9095"}]: dispatch 2024-03-24T11:15:19.049 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:18 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:19.049 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:18 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:15:19.049 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:18 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:15:19.050 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:18 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:15:19.050 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:18 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:19.050 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:18 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:15:19.050 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:18 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:15:19.050 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:18 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:15:19.050 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:18 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:19.050 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:18 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:19.050 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:18 smithi139 podman[43155]: 2024-03-24 11:15:18.697888938 +0000 UTC m=+0.349975976 container init 29c5ca3f2ecc25784888c73321b74bcbdc5aed758f5937467948bd71a1497142 (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-24T11:15:19.050 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:18 smithi139 podman[43155]: 2024-03-24 11:15:18.700541892 +0000 UTC m=+0.352628929 container start 29c5ca3f2ecc25784888c73321b74bcbdc5aed758f5937467948bd71a1497142 (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-24T11:15:19.051 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:18 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43186]: ts=2024-03-24T11:15:18.723Z caller=main.go:475 level=info msg="No time or size retention was set so using the default time retention" duration=15d 2024-03-24T11:15:19.051 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:18 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43186]: ts=2024-03-24T11:15:18.723Z caller=main.go:512 level=info msg="Starting Prometheus" version="(version=2.33.4, branch=HEAD, revision=83032011a5d3e6102624fe58241a374a7201fee8)" 2024-03-24T11:15:19.051 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:18 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43186]: ts=2024-03-24T11:15:18.723Z caller=main.go:517 level=info build_context="(go=go1.17.7, user=root@d13bf69e7be8, date=20220222-16:51:28)" 2024-03-24T11:15:19.051 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:18 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43186]: ts=2024-03-24T11:15:18.723Z caller=main.go:518 level=info host_details="(Linux 5.14.0-430.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024 x86_64 smithi139 (none))" 2024-03-24T11:15:19.051 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:18 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43186]: ts=2024-03-24T11:15:18.724Z caller=main.go:519 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-03-24T11:15:19.051 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:18 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43186]: ts=2024-03-24T11:15:18.724Z caller=main.go:520 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-03-24T11:15:19.051 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:18 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43186]: ts=2024-03-24T11:15:18.725Z caller=web.go:570 level=info component=web msg="Start listening for connections" address=:9095 2024-03-24T11:15:19.051 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:18 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43186]: ts=2024-03-24T11:15:18.726Z caller=main.go:923 level=info msg="Starting TSDB ..." 2024-03-24T11:15:19.051 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:18 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43186]: ts=2024-03-24T11:15:18.727Z caller=tls_config.go:195 level=info component=web msg="TLS is disabled." http2=false 2024-03-24T11:15:19.051 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:18 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43186]: ts=2024-03-24T11:15:18.728Z caller=head.go:493 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-03-24T11:15:19.051 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:18 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43186]: ts=2024-03-24T11:15:18.728Z caller=head.go:527 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=2.249µs 2024-03-24T11:15:19.051 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:18 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43186]: ts=2024-03-24T11:15:18.728Z caller=head.go:533 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-03-24T11:15:19.051 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:18 smithi139 bash[43155]: 29c5ca3f2ecc25784888c73321b74bcbdc5aed758f5937467948bd71a1497142 2024-03-24T11:15:19.051 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:18 smithi139 systemd[1]: Started Ceph prometheus.a for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:15:19.051 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:18 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43186]: ts=2024-03-24T11:15:18.921Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-03-24T11:15:19.051 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:18 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43186]: ts=2024-03-24T11:15:18.921Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-03-24T11:15:19.052 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:18 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43186]: ts=2024-03-24T11:15:18.921Z caller=head.go:610 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=34.909µs wal_replay_duration=192.52005ms total_replay_duration=192.599086ms 2024-03-24T11:15:19.052 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:18 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43186]: ts=2024-03-24T11:15:18.923Z caller=main.go:944 level=info fs_type=XFS_SUPER_MAGIC 2024-03-24T11:15:19.052 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:18 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43186]: ts=2024-03-24T11:15:18.923Z caller=main.go:947 level=info msg="TSDB started" 2024-03-24T11:15:19.052 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:18 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43186]: ts=2024-03-24T11:15:18.923Z caller=main.go:1128 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-03-24T11:15:19.052 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:18 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43186]: ts=2024-03-24T11:15:18.938Z caller=main.go:1165 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=15.012535ms db_storage=712ns remote_storage=1.184µs web_handler=740ns query_engine=575ns scrape=354.48µs scrape_sd=26.649µs notify=792ns notify_sd=1.322µs rules=14.387801ms 2024-03-24T11:15:19.052 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:18 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43186]: ts=2024-03-24T11:15:18.938Z caller=main.go:896 level=info msg="Server is ready to receive web requests." 2024-03-24T11:15:19.077 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:15:18 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.62 - - [24/Mar/2024:11:15:18] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:15:19.078 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:15:19 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.62 - - [24/Mar/2024:11:15:19] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:15:19.078 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:18 smithi062 ceph-mon[29666]: pgmap v18: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:19.078 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:18 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:19.078 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:18 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:19.078 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:18 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T11:15:19.078 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:18 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://172.21.15.139:9095"}]: dispatch 2024-03-24T11:15:19.078 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:18 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:19.078 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:18 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:15:19.078 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:18 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:15:19.078 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:18 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:15:19.079 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:18 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:19.079 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:18 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:15:19.079 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:18 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:15:19.079 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:18 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:15:19.079 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:18 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:19.079 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:18 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:19.079 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:18 smithi062 ceph-mon[24799]: pgmap v18: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:19.079 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:18 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:19.079 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:18 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:19.079 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:18 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T11:15:19.079 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:18 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://172.21.15.139:9095"}]: dispatch 2024-03-24T11:15:19.079 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:18 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:19.079 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:18 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:15:19.080 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:18 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:15:19.080 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:18 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:15:19.080 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:18 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:19.080 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:18 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:15:19.080 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:18 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:15:19.080 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:18 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:15:19.080 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:18 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:19.080 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:18 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:19.971 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:19 smithi139 ceph-mon[25719]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T11:15:19.971 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:19 smithi139 ceph-mon[25719]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://172.21.15.139:9095"}]: dispatch 2024-03-24T11:15:19.971 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:19 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:15:19.971 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:19 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:15:19.971 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:19 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:19.972 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:19 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:19.972 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:19 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:15:19.972 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:19 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:15:19.972 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:19 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:15:19.972 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:19 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:20.191 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:19 smithi062 ceph-mon[29666]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T11:15:20.191 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:19 smithi062 ceph-mon[29666]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://172.21.15.139:9095"}]: dispatch 2024-03-24T11:15:20.191 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:19 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:15:20.191 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:19 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:15:20.191 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:19 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:20.191 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:19 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:20.191 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:19 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:15:20.191 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:19 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:15:20.191 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:19 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:15:20.191 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:19 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:20.192 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:19 smithi062 ceph-mon[24799]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T11:15:20.192 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:19 smithi062 ceph-mon[24799]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://172.21.15.139:9095"}]: dispatch 2024-03-24T11:15:20.192 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:19 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:15:20.192 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:19 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:15:20.192 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:19 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:20.192 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:19 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:20.192 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:19 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:15:20.192 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:19 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:15:20.192 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:19 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:15:20.192 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:19 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:21.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:20 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:15:21.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:20 smithi139 ceph-mon[25719]: Reconfiguring prometheus.a (dependencies changed)... 2024-03-24T11:15:21.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:20 smithi139 ceph-mon[25719]: Reconfiguring daemon prometheus.a on smithi139 2024-03-24T11:15:21.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:20 smithi139 ceph-mon[25719]: pgmap v19: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:21.170 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:20 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:15:21.170 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:20 smithi062 ceph-mon[24799]: Reconfiguring prometheus.a (dependencies changed)... 2024-03-24T11:15:21.170 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:20 smithi062 ceph-mon[24799]: Reconfiguring daemon prometheus.a on smithi139 2024-03-24T11:15:21.170 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:20 smithi062 ceph-mon[24799]: pgmap v19: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:21.171 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:20 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:15:21.171 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:20 smithi062 ceph-mon[29666]: Reconfiguring prometheus.a (dependencies changed)... 2024-03-24T11:15:21.171 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:20 smithi062 ceph-mon[29666]: Reconfiguring daemon prometheus.a on smithi139 2024-03-24T11:15:21.171 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:20 smithi062 ceph-mon[29666]: pgmap v19: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:22.044 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:15:21 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.139 - - [24/Mar/2024:11:15:21] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:15:22.311 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:15:22 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.139 - - [24/Mar/2024:11:15:22] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:15:22.338 INFO:teuthology.orchestra.run.smithi062.stdout:111669149727 2024-03-24T11:15:22.339 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-24T11:15:22.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:22 smithi062 ceph-mon[29666]: pgmap v20: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:22.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:22 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:22.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:22 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:22.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:22 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:22.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:22 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:22.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:22 smithi062 ceph-mon[24799]: pgmap v20: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:22.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:22 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:22.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:22 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:22.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:22 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:22.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:22 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:23.347 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:22 smithi139 ceph-mon[25719]: pgmap v20: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:23.347 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:22 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:23.347 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:22 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:23.347 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:22 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:23.347 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:22 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:23.888 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T11:15:24.936 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:24 smithi139 ceph-mon[25719]: pgmap v21: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:24.937 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:24 smithi139 systemd[1]: Stopping Ceph prometheus.a for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:15:24.937 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:24 smithi139 bash[43740]: Error: no container with name or ID "ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus.a" found: no such container 2024-03-24T11:15:25.023 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:24 smithi062 ceph-mon[29666]: pgmap v21: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:25.024 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:24 smithi062 ceph-mon[24799]: pgmap v21: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:25.212 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:24 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43186]: ts=2024-03-24T11:15:24.936Z caller=main.go:775 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-03-24T11:15:25.212 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:24 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43186]: ts=2024-03-24T11:15:24.936Z caller=main.go:798 level=info msg="Stopping scrape discovery manager..." 2024-03-24T11:15:25.212 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:24 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43186]: ts=2024-03-24T11:15:24.936Z caller=main.go:812 level=info msg="Stopping notify discovery manager..." 2024-03-24T11:15:25.212 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:24 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43186]: ts=2024-03-24T11:15:24.936Z caller=main.go:834 level=info msg="Stopping scrape manager..." 2024-03-24T11:15:25.212 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:24 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43186]: ts=2024-03-24T11:15:24.936Z caller=main.go:808 level=info msg="Notify discovery manager stopped" 2024-03-24T11:15:25.213 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:24 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43186]: ts=2024-03-24T11:15:24.936Z caller=main.go:794 level=info msg="Scrape discovery manager stopped" 2024-03-24T11:15:25.213 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:24 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43186]: ts=2024-03-24T11:15:24.936Z caller=manager.go:945 level=info component="rule manager" msg="Stopping rule manager..." 2024-03-24T11:15:25.213 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:24 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43186]: ts=2024-03-24T11:15:24.936Z caller=main.go:828 level=info msg="Scrape manager stopped" 2024-03-24T11:15:25.213 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:24 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43186]: ts=2024-03-24T11:15:24.936Z caller=manager.go:955 level=info component="rule manager" msg="Rule manager stopped" 2024-03-24T11:15:25.213 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:24 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43186]: ts=2024-03-24T11:15:24.936Z caller=notifier.go:600 level=info component=notifier msg="Stopping notification manager..." 2024-03-24T11:15:25.213 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:24 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43186]: ts=2024-03-24T11:15:24.936Z caller=main.go:1054 level=info msg="Notifier manager stopped" 2024-03-24T11:15:25.213 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:24 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43186]: ts=2024-03-24T11:15:24.937Z caller=main.go:1066 level=info msg="See you next time!" 2024-03-24T11:15:25.213 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:24 smithi139 podman[43753]: 2024-03-24 11:15:24.946583328 +0000 UTC m=+0.116172341 container died 29c5ca3f2ecc25784888c73321b74bcbdc5aed758f5937467948bd71a1497142 (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-24T11:15:25.468 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:25 smithi139 podman[43753]: 2024-03-24 11:15:25.286060715 +0000 UTC m=+0.455649793 container cleanup 29c5ca3f2ecc25784888c73321b74bcbdc5aed758f5937467948bd71a1497142 (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-24T11:15:25.468 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:25 smithi139 bash[43753]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a 2024-03-24T11:15:25.468 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:25 smithi139 podman[43768]: 2024-03-24 11:15:25.461796694 +0000 UTC m=+0.516932175 container remove 29c5ca3f2ecc25784888c73321b74bcbdc5aed758f5937467948bd71a1497142 (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-24T11:15:25.481 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T11:15:25.737 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:25 smithi139 bash[43780]: Error: no container with name or ID "ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus.a" found: no such container 2024-03-24T11:15:25.738 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:25 smithi139 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@prometheus.a.service: Deactivated successfully. 2024-03-24T11:15:25.738 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:25 smithi139 systemd[1]: Stopped Ceph prometheus.a for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:15:25.738 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:25 smithi139 systemd[1]: Starting Ceph prometheus.a for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:15:25.738 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:25 smithi139 podman[43850]: 2024-03-24 11:15:25.612087686 +0000 UTC m=+0.018640211 image pull 514e6a882f6e74806a5856468489eeff8d7106095557578da96935e4d0ba4d9d quay.io/prometheus/prometheus:v2.33.4 2024-03-24T11:15:25.738 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:25 smithi139 podman[43850]: 2024-03-24 11:15:25.73781222 +0000 UTC m=+0.144364786 container create c1d79fd6d6f1fe40241d869930eabf531f3134eb22a713215b0893fe6f893692 (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-24T11:15:25.901 INFO:teuthology.orchestra.run.smithi062.stdout:34359738413 2024-03-24T11:15:25.902 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-24T11:15:26.006 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:25 smithi139 podman[43850]: 2024-03-24 11:15:25.981481545 +0000 UTC m=+0.388034069 container init c1d79fd6d6f1fe40241d869930eabf531f3134eb22a713215b0893fe6f893692 (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-24T11:15:26.007 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:25 smithi139 podman[43850]: 2024-03-24 11:15:25.984086773 +0000 UTC m=+0.390639297 container start c1d79fd6d6f1fe40241d869930eabf531f3134eb22a713215b0893fe6f893692 (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-24T11:15:26.358 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:26 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43865]: ts=2024-03-24T11:15:26.007Z caller=main.go:475 level=info msg="No time or size retention was set so using the default time retention" duration=15d 2024-03-24T11:15:26.358 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:26 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43865]: ts=2024-03-24T11:15:26.007Z caller=main.go:512 level=info msg="Starting Prometheus" version="(version=2.33.4, branch=HEAD, revision=83032011a5d3e6102624fe58241a374a7201fee8)" 2024-03-24T11:15:26.358 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:26 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43865]: ts=2024-03-24T11:15:26.007Z caller=main.go:517 level=info build_context="(go=go1.17.7, user=root@d13bf69e7be8, date=20220222-16:51:28)" 2024-03-24T11:15:26.358 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:26 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43865]: ts=2024-03-24T11:15:26.007Z caller=main.go:518 level=info host_details="(Linux 5.14.0-430.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024 x86_64 smithi139 (none))" 2024-03-24T11:15:26.358 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:26 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43865]: ts=2024-03-24T11:15:26.007Z caller=main.go:519 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-03-24T11:15:26.359 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:26 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43865]: ts=2024-03-24T11:15:26.007Z caller=main.go:520 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-03-24T11:15:26.359 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:26 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43865]: ts=2024-03-24T11:15:26.009Z caller=web.go:570 level=info component=web msg="Start listening for connections" address=:9095 2024-03-24T11:15:26.359 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:26 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43865]: ts=2024-03-24T11:15:26.009Z caller=main.go:923 level=info msg="Starting TSDB ..." 2024-03-24T11:15:26.359 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:26 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43865]: ts=2024-03-24T11:15:26.010Z caller=tls_config.go:195 level=info component=web msg="TLS is disabled." http2=false 2024-03-24T11:15:26.359 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:26 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43865]: ts=2024-03-24T11:15:26.012Z caller=head.go:493 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-03-24T11:15:26.359 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:26 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43865]: ts=2024-03-24T11:15:26.012Z caller=head.go:527 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=3.034µs 2024-03-24T11:15:26.359 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:26 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43865]: ts=2024-03-24T11:15:26.012Z caller=head.go:533 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-03-24T11:15:26.359 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:26 smithi139 bash[43850]: c1d79fd6d6f1fe40241d869930eabf531f3134eb22a713215b0893fe6f893692 2024-03-24T11:15:26.359 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:26 smithi139 systemd[1]: Started Ceph prometheus.a for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:15:26.359 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:26 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43865]: ts=2024-03-24T11:15:26.177Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=2 2024-03-24T11:15:26.359 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:26 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43865]: ts=2024-03-24T11:15:26.178Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=2 2024-03-24T11:15:26.359 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:26 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43865]: ts=2024-03-24T11:15:26.178Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=2 maxSegment=2 2024-03-24T11:15:26.360 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:26 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43865]: ts=2024-03-24T11:15:26.178Z caller=head.go:610 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=52.928µs wal_replay_duration=165.697117ms total_replay_duration=165.815094ms 2024-03-24T11:15:26.360 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:26 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43865]: ts=2024-03-24T11:15:26.180Z caller=main.go:944 level=info fs_type=XFS_SUPER_MAGIC 2024-03-24T11:15:26.360 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:26 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43865]: ts=2024-03-24T11:15:26.180Z caller=main.go:947 level=info msg="TSDB started" 2024-03-24T11:15:26.360 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:26 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43865]: ts=2024-03-24T11:15:26.180Z caller=main.go:1128 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-03-24T11:15:26.360 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:26 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43865]: ts=2024-03-24T11:15:26.194Z caller=main.go:1165 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=14.824127ms db_storage=570ns remote_storage=2.116µs web_handler=303ns query_engine=690ns scrape=360.564µs scrape_sd=22.098µs notify=739ns notify_sd=1.944µs rules=14.248041ms 2024-03-24T11:15:26.360 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:15:26 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43865]: ts=2024-03-24T11:15:26.195Z caller=main.go:896 level=info msg="Server is ready to receive web requests." 2024-03-24T11:15:26.846 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T11:15:26.847 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T11:15:27.147 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:26 smithi062 ceph-mon[24799]: pgmap v22: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:27.147 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:26 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:27.147 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:26 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T11:15:27.147 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:26 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:15:27.147 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:26 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:15:27.147 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:26 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:15:27.147 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:26 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:27.147 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:26 smithi062 ceph-mon[29666]: pgmap v22: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:27.148 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:26 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:27.148 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:26 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T11:15:27.148 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:26 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:15:27.148 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:26 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:15:27.148 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:26 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:15:27.148 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:26 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:27.339 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:26 smithi139 ceph-mon[25719]: pgmap v22: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:27.339 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:26 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:27.339 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:26 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T11:15:27.339 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:26 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:15:27.339 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:26 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:15:27.339 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:26 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:15:27.339 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:26 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:28.134 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:27 smithi062 ceph-mon[24799]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T11:15:28.134 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:27 smithi062 ceph-mon[24799]: Deploying daemon alertmanager.a on smithi062 2024-03-24T11:15:28.134 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:27 smithi062 ceph-mon[29666]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T11:15:28.135 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:27 smithi062 ceph-mon[29666]: Deploying daemon alertmanager.a on smithi062 2024-03-24T11:15:28.234 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:27 smithi139 ceph-mon[25719]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T11:15:28.234 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:27 smithi139 ceph-mon[25719]: Deploying daemon alertmanager.a on smithi062 2024-03-24T11:15:28.612 INFO:teuthology.orchestra.run.smithi062.stdout:167503724566 2024-03-24T11:15:28.612 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-24T11:15:29.195 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:28 smithi062 ceph-mon[24799]: pgmap v23: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:29.195 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:28 smithi062 ceph-mon[29666]: pgmap v23: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:29.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:28 smithi139 ceph-mon[25719]: pgmap v23: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:30.331 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T11:15:30.356 INFO:teuthology.orchestra.run.smithi062.stdout:133143986204 2024-03-24T11:15:30.356 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-24T11:15:31.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:30 smithi062 ceph-mon[24799]: pgmap v24: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:31.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:30 smithi062 ceph-mon[29666]: pgmap v24: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:31.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:30 smithi139 ceph-mon[25719]: pgmap v24: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:33.160 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:32 smithi139 ceph-mon[25719]: pgmap v25: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:33.200 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:32 smithi062 ceph-mon[24799]: pgmap v25: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:33.200 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:32 smithi062 ceph-mon[29666]: pgmap v25: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:35.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:34 smithi062 ceph-mon[24799]: pgmap v26: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:35.172 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:34 smithi062 ceph-mon[29666]: pgmap v26: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:35.255 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:34 smithi139 ceph-mon[25719]: pgmap v26: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:35.978 INFO:teuthology.orchestra.run.smithi062.stdout:201863462933 2024-03-24T11:15:35.979 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-03-24T11:15:36.417 INFO:teuthology.orchestra.run.smithi062.stdout:77309411367 2024-03-24T11:15:36.417 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-24T11:15:37.111 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:36 smithi062 ceph-mon[24799]: pgmap v27: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:37.111 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:36 smithi062 ceph-mon[29666]: pgmap v27: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:37.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:36 smithi139 ceph-mon[25719]: pgmap v27: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:38.976 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:38 smithi062 ceph-mon[24799]: pgmap v28: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:38.977 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:15:38 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:15:38] "GET /metrics HTTP/1.1" 200 191484 "" "Prometheus/2.33.4" 2024-03-24T11:15:38.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:38 smithi062 ceph-mon[29666]: pgmap v28: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:39.176 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T11:15:39.218 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:38 smithi139 ceph-mon[25719]: pgmap v28: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:39.284 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:15:39 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.62 - - [24/Mar/2024:11:15:39] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:15:39.308 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T11:15:39.309 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T11:15:39.309 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T11:15:39.481 INFO:teuthology.orchestra.run.smithi062.stdout:34359738416 2024-03-24T11:15:39.727 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:15:39 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.139 - - [24/Mar/2024:11:15:39] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:15:39.760 INFO:teuthology.orchestra.run.smithi062.stdout:55834574891 2024-03-24T11:15:39.761 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-24T11:15:40.274 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:40 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:40.274 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:40 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:40.275 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:40 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/1940739695' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-24T11:15:40.275 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:40 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:40.275 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:40 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:40.275 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:40 smithi062 ceph-mon[24799]: pgmap v29: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:40.275 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:40 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:40.275 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:40 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:40.275 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:40 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/1940739695' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-24T11:15:40.275 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:40 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:40.275 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:40 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:40.275 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:40 smithi062 ceph-mon[29666]: pgmap v29: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:40.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:40 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:40.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:40 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:40.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:40 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/1940739695' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-24T11:15:40.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:40 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:40.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:40 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:40.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:40 smithi139 ceph-mon[25719]: pgmap v29: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:41.108 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:15:40 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:15:40] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-03-24T11:15:42.989 INFO:teuthology.orchestra.run.smithi062.stdout:111669149731 2024-03-24T11:15:43.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:42 smithi062 ceph-mon[24799]: pgmap v30: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:43.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:42 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/2732459606' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-24T11:15:43.126 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:42 smithi062 ceph-mon[29666]: pgmap v30: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:43.126 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:42 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/2732459606' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-24T11:15:43.209 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:42 smithi139 ceph-mon[25719]: pgmap v30: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:43.210 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:42 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/2732459606' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-24T11:15:44.471 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738413 got 34359738416 for osd.0 2024-03-24T11:15:44.471 DEBUG:teuthology.parallel:result is None 2024-03-24T11:15:44.829 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149727 got 111669149731 for osd.3 2024-03-24T11:15:44.830 DEBUG:teuthology.parallel:result is None 2024-03-24T11:15:45.132 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:44 smithi062 ceph-mon[24799]: pgmap v31: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:45.132 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:44 smithi062 ceph-mon[29666]: pgmap v31: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:45.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:44 smithi139 ceph-mon[25719]: pgmap v31: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:47.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:46 smithi062 ceph-mon[24799]: pgmap v32: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:47.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:46 smithi062 ceph-mon[29666]: pgmap v32: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:47.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:46 smithi139 ceph-mon[25719]: pgmap v32: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:48.507 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T11:15:48.890 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:15:48 smithi062 systemd[1]: Starting Ceph alertmanager.a for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:15:49.004 INFO:teuthology.orchestra.run.smithi062.stdout:236223201297 2024-03-24T11:15:49.005 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-03-24T11:15:49.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:48 smithi062 ceph-mon[24799]: pgmap v33: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:49.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:48 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-24T11:15:49.217 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:48 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-24T11:15:49.218 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:15:48 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:15:48] "GET /metrics HTTP/1.1" 200 191495 "" "Prometheus/2.33.4" 2024-03-24T11:15:49.218 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:48 smithi062 ceph-mon[29666]: pgmap v33: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:49.218 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:48 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-24T11:15:49.218 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:48 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-24T11:15:49.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:48 smithi139 ceph-mon[25719]: pgmap v33: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:49.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:48 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-24T11:15:49.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:48 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-24T11:15:49.412 INFO:teuthology.orchestra.run.smithi062.stdout:133143986208 2024-03-24T11:15:50.091 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:49 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/3276558416' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-24T11:15:50.092 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:49 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/3276558416' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-24T11:15:50.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:49 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/3276558416' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-24T11:15:50.917 INFO:teuthology.orchestra.run.smithi062.stdout:201863462936 2024-03-24T11:15:51.097 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:50 smithi062 ceph-mon[24799]: pgmap v34: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:51.098 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:50 smithi062 ceph-mon[29666]: pgmap v34: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:51.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:50 smithi139 ceph-mon[25719]: pgmap v34: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:51.108 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:15:50 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:15:50] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-03-24T11:15:51.119 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986204 got 133143986208 for osd.4 2024-03-24T11:15:51.119 DEBUG:teuthology.parallel:result is None 2024-03-24T11:15:51.351 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T11:15:52.068 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:51 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/2490278318' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-24T11:15:52.068 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:51 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/2490278318' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-24T11:15:52.231 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:51 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/2490278318' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-24T11:15:52.734 INFO:teuthology.orchestra.run.smithi062.stdout:167503724571 2024-03-24T11:15:52.976 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:52 smithi062 ceph-mon[24799]: pgmap v35: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:52.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:52 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/1628251246' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-24T11:15:52.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:52 smithi062 ceph-mon[29666]: pgmap v35: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:52.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:52 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/1628251246' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-24T11:15:52.993 INFO:teuthology.orchestra.run.smithi062.stdout:77309411370 2024-03-24T11:15:53.058 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:52 smithi139 ceph-mon[25719]: pgmap v35: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:53.058 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:52 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/1628251246' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-24T11:15:53.796 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724566 got 167503724571 for osd.5 2024-03-24T11:15:53.797 DEBUG:teuthology.parallel:result is None 2024-03-24T11:15:54.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:53 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/1087700838' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-24T11:15:54.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:53 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:54.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:53 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:54.162 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:15:53 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.139 - - [24/Mar/2024:11:15:53] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:15:54.163 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:53 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/1087700838' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-24T11:15:54.163 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:53 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:54.163 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:53 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:54.163 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:53 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/1087700838' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-24T11:15:54.163 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:53 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:54.163 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:53 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:15:54.187 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411367 got 77309411370 for osd.2 2024-03-24T11:15:54.188 DEBUG:teuthology.parallel:result is None 2024-03-24T11:15:55.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:54 smithi139 ceph-mon[25719]: pgmap v36: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:55.226 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:54 smithi062 ceph-mon[29666]: pgmap v36: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:55.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:54 smithi062 ceph-mon[24799]: pgmap v36: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:56.055 INFO:teuthology.orchestra.run.smithi062.stdout:236223201299 2024-03-24T11:15:56.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:56 smithi062 ceph-mon[24799]: pgmap v37: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:56.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:56 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/636522549' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-24T11:15:56.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:56 smithi062 ceph-mon[29666]: pgmap v37: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:56.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:56 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/636522549' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-24T11:15:57.197 INFO:teuthology.orchestra.run.smithi062.stdout:55834574895 2024-03-24T11:15:57.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:56 smithi139 ceph-mon[25719]: pgmap v37: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:57.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:56 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/636522549' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-24T11:15:57.592 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462933 got 201863462936 for osd.6 2024-03-24T11:15:57.592 DEBUG:teuthology.parallel:result is None 2024-03-24T11:15:58.190 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:57 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/1900494809' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-24T11:15:58.190 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:15:57 smithi062 ceph-mon[24799]: pgmap v38: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:58.190 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:57 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/1900494809' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-24T11:15:58.190 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:15:57 smithi062 ceph-mon[29666]: pgmap v38: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:58.214 INFO:tasks.cephadm.ceph_manager.ceph:need seq 236223201297 got 236223201299 for osd.7 2024-03-24T11:15:58.215 DEBUG:teuthology.parallel:result is None 2024-03-24T11:15:58.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:57 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/1900494809' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-24T11:15:58.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:15:57 smithi139 ceph-mon[25719]: pgmap v38: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:15:58.601 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574891 got 55834574895 for osd.1 2024-03-24T11:15:58.602 DEBUG:teuthology.parallel:result is None 2024-03-24T11:15:58.602 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-24T11:15:58.602 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph pg dump --format=json 2024-03-24T11:15:58.890 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:15:58 smithi062 podman[49279]: 2024-03-24 11:15:58.718569535 +0000 UTC m=+0.097144575 volume create 2e0075e627bc6969d7b8c067893aa716bc55f47f7b2765814d680ff1b0af0be5 2024-03-24T11:15:58.891 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:15:58 smithi062 podman[49279]: 2024-03-24 11:15:58.645078678 +0000 UTC m=+0.023653727 image pull ba2b418f427c0636d654de8757e830c80168e76482bcc46bb2138e569d6c91d4 quay.io/prometheus/alertmanager:v0.23.0 2024-03-24T11:15:58.891 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:15:58 smithi062 podman[49279]: 2024-03-24 11:15:58.79395126 +0000 UTC m=+0.172526308 container create 69b54ef5350762804269ebc2f0d9acd593b44efd48a9ec7ed2e33338af7512c6 (image=quay.io/prometheus/alertmanager:v0.23.0, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-24T11:15:59.155 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:15:58 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:15:58] "GET /metrics HTTP/1.1" 200 191495 "" "Prometheus/2.33.4" 2024-03-24T11:15:59.155 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:15:59 smithi062 podman[49279]: 2024-03-24 11:15:59.071942205 +0000 UTC m=+0.450517246 container init 69b54ef5350762804269ebc2f0d9acd593b44efd48a9ec7ed2e33338af7512c6 (image=quay.io/prometheus/alertmanager:v0.23.0, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-24T11:15:59.155 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:15:59 smithi062 podman[49279]: 2024-03-24 11:15:59.074377583 +0000 UTC m=+0.452952633 container start 69b54ef5350762804269ebc2f0d9acd593b44efd48a9ec7ed2e33338af7512c6 (image=quay.io/prometheus/alertmanager:v0.23.0, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-24T11:15:59.155 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:15:59 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[49311]: level=info ts=2024-03-24T11:15:59.084Z caller=main.go:225 msg="Starting Alertmanager" version="(version=0.23.0, branch=HEAD, revision=61046b17771a57cfd4c4a51be370ab930a4d7d54)" 2024-03-24T11:15:59.156 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:15:59 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[49311]: level=info ts=2024-03-24T11:15:59.084Z caller=main.go:226 build_context="(go=go1.16.7, user=root@e21a959be8d2, date=20210825-10:48:55)" 2024-03-24T11:15:59.156 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:15:59 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[49311]: level=info ts=2024-03-24T11:15:59.085Z caller=cluster.go:184 component=cluster msg="setting advertise address explicitly" addr=172.21.15.62 port=9094 2024-03-24T11:15:59.156 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:15:59 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[49311]: level=info ts=2024-03-24T11:15:59.086Z caller=cluster.go:671 component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-03-24T11:15:59.156 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:15:59 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[49311]: level=info ts=2024-03-24T11:15:59.111Z caller=coordinator.go:113 component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-03-24T11:15:59.156 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:15:59 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[49311]: level=info ts=2024-03-24T11:15:59.111Z caller=coordinator.go:126 component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-03-24T11:15:59.156 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:15:59 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[49311]: level=info ts=2024-03-24T11:15:59.113Z caller=main.go:518 msg=Listening address=:9093 2024-03-24T11:15:59.156 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:15:59 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[49311]: level=info ts=2024-03-24T11:15:59.114Z caller=tls_config.go:191 msg="TLS is disabled." http2=false 2024-03-24T11:15:59.156 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:15:59 smithi062 bash[49279]: 69b54ef5350762804269ebc2f0d9acd593b44efd48a9ec7ed2e33338af7512c6 2024-03-24T11:15:59.476 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:15:59 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.62 - - [24/Mar/2024:11:15:59] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:15:59.477 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:15:59 smithi062 systemd[1]: Started Ceph alertmanager.a for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:15:59.507 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T11:16:00.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:00 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:00.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:00 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:00.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:00 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:00.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:00 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-24T11:16:00.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:00 smithi062 ceph-mon[24799]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-24T11:16:00.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:00 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:00.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:00 smithi062 ceph-mon[24799]: Deploying daemon grafana.a on smithi139 2024-03-24T11:16:00.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:00 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:00.371 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:00 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:00.371 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:00 smithi062 ceph-mon[24799]: pgmap v39: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:16:00.371 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:00 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:00.371 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:00 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:00.371 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:00 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:00.371 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:00 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-24T11:16:00.371 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:00 smithi062 ceph-mon[29666]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-24T11:16:00.371 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:00 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:00.371 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:00 smithi062 ceph-mon[29666]: Deploying daemon grafana.a on smithi139 2024-03-24T11:16:00.371 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:00 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:00.372 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:00 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:00.372 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:00 smithi062 ceph-mon[29666]: pgmap v39: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:16:00.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:00 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:00.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:00 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:00.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:00 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:00.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:00 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-24T11:16:00.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:00 smithi139 ceph-mon[25719]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-24T11:16:00.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:00 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:00.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:00 smithi139 ceph-mon[25719]: Deploying daemon grafana.a on smithi139 2024-03-24T11:16:00.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:00 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:00.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:00 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:00.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:00 smithi139 ceph-mon[25719]: pgmap v39: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:16:01.107 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:16:00 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:16:00] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-03-24T11:16:01.395 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:16:01 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[49311]: level=info ts=2024-03-24T11:16:01.087Z caller=cluster.go:696 component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000905561s 2024-03-24T11:16:03.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:02 smithi062 ceph-mon[24799]: pgmap v40: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:16:03.124 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:02 smithi062 ceph-mon[29666]: pgmap v40: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:16:03.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:02 smithi139 ceph-mon[25719]: pgmap v40: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:16:03.407 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-24T11:16:03.409 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-03-24T11:16:04.005 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:03 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:04.005 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:03 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:04.060 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2024-03-24T11:16:01.845361+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1586784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":8,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":749961216,"kb_used":48908,"kb_used_data":6156,"kb_used_omap":0,"kb_used_meta":42688,"kb_avail":749912308,"statfs":{"total":767960285184,"available":767910203392,"internally_reserved":0,"allocated":6303744,"data_stored":3511211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":43712512},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002326"},"pg_stats":[{"pgid":"1.0","version":"60'128","reported_seq":267,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-03-24T11:14:47.951443+0000","last_change":"2024-03-24T11:14:31.858037+0000","last_active":"2024-03-24T11:14:47.951443+0000","last_peered":"2024-03-24T11:14:47.951443+0000","last_clean":"2024-03-24T11:14:47.951443+0000","last_became_active":"2024-03-24T11:14:31.857616+0000","last_became_peered":"2024-03-24T11:14:31.857616+0000","last_unstale":"2024-03-24T11:14:47.951443+0000","last_undegraded":"2024-03-24T11:14:47.951443+0000","last_fullsized":"2024-03-24T11:14:47.951443+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T11:12:32.424867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T11:12:32.424867+0000","last_clean_scrub_stamp":"2024-03-24T11:12:32.424867+0000","objects_scrubbed":0,"log_size":128,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:08:32.528538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1586784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1586784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":55,"seq":236223201300,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6096,"kb_used_data":1096,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":93739056,"statfs":{"total":95995035648,"available":95988793344,"internally_reserved":0,"allocated":1122304,"data_stored":769588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Mar 24 11:15:28 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.58299999999999996}]},{"osd":1,"last update":"Sun Mar 24 11:15:28 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.53800000000000003}]},{"osd":2,"last update":"Sun Mar 24 11:15:28 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.498},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.57099999999999995}]},{"osd":3,"last update":"Sun Mar 24 11:15:28 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.439},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.60299999999999998}]},{"osd":4,"last update":"Sun Mar 24 11:15:28 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.55500000000000005}]},{"osd":5,"last update":"Sun Mar 24 11:15:28 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.47999999999999998}]},{"osd":6,"last update":"Sun Mar 24 11:15:28 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.59399999999999997}]}]},{"osd":6,"up_from":47,"seq":201863462938,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6152,"kb_used_data":1088,"kb_used_omap":0,"kb_used_meta":5056,"kb_avail":93739000,"statfs":{"total":95995035648,"available":95988736000,"internally_reserved":0,"allocated":1114112,"data_stored":769000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5177344},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Mar 24 11:15:03 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.61899999999999999}]},{"osd":1,"last update":"Sun Mar 24 11:15:03 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.69999999999999996}]},{"osd":2,"last update":"Sun Mar 24 11:15:03 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.64300000000000002}]},{"osd":3,"last update":"Sun Mar 24 11:15:03 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.627},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.72799999999999998}]},{"osd":4,"last update":"Sun Mar 24 11:15:03 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.438},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.46200000000000002}]},{"osd":5,"last update":"Sun Mar 24 11:15:03 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.505},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.48999999999999999}]},{"osd":7,"last update":"Sun Mar 24 11:15:30 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.53400000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":6344,"kb_used_data":576,"kb_used_omap":0,"kb_used_meta":5760,"kb_avail":93738808,"statfs":{"total":95995035648,"available":95988539392,"internally_reserved":0,"allocated":589824,"data_stored":240660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5898240},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Mar 24 11:15:13 2024","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.32900000000000001,"15min":0.31900000000000001},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.33700000000000002,"15min":0.318},"min":{"1min":0.247,"5min":0.215,"15min":0.215},"max":{"1min":0.57099999999999995,"5min":0.57799999999999996,"15min":0.57799999999999996},"last":0.41399999999999998}]},{"osd":2,"last update":"Sun Mar 24 11:15:34 2024","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.371,"15min":0.36599999999999999},"min":{"1min":0.253,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.56899999999999995,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.434,"15min":0.42099999999999999},"min":{"1min":0.34200000000000003,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.73599999999999999,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.50600000000000001}]},{"osd":3,"last update":"Sun Mar 24 11:15:02 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.44,"15min":0.443},"min":{"1min":0.316,"5min":0.31,"15min":0.31},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.42499999999999999,"15min":0.42899999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"last":0.38800000000000001}]},{"osd":4,"last update":"Sun Mar 24 11:15:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.58699999999999997,"15min":0.59699999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.76800000000000002,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.67600000000000005,"15min":0.70199999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.78100000000000003,"5min":3.4460000000000002,"15min":3.4460000000000002},"last":0.66900000000000004}]},{"osd":5,"last update":"Sun Mar 24 11:15:39 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.55000000000000004,"15min":0.55400000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72099999999999997,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56999999999999995,"15min":0.57499999999999996},"min":{"1min":0.41799999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.56899999999999995}]},{"osd":6,"last update":"Sun Mar 24 11:15:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.61199999999999999}]},{"osd":7,"last update":"Sun Mar 24 11:15:28 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":5.2210000000000001,"5min":5.2210000000000001,"15min":5.2210000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.63900000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738421,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6928,"kb_used_data":1096,"kb_used_omap":0,"kb_used_meta":5824,"kb_avail":93738224,"statfs":{"total":95995035648,"available":95987941376,"internally_reserved":0,"allocated":1122304,"data_stored":769588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5963776},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Mar 24 11:15:13 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.35699999999999998,"15min":0.33700000000000002},"min":{"1min":0.247,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.58699999999999997,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.377,"15min":0.35599999999999998},"min":{"1min":0.27800000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.34799999999999998}]},{"osd":2,"last update":"Sun Mar 24 11:15:36 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.39100000000000001,"15min":0.378},"min":{"1min":0.28100000000000003,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.39000000000000001,"15min":0.38},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.61199999999999999,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.254}]},{"osd":3,"last update":"Sun Mar 24 11:15:00 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.45800000000000002,"15min":0.45200000000000001},"min":{"1min":0.34699999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.64500000000000002,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.45200000000000001,"15min":0.45000000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.30399999999999999}]},{"osd":4,"last update":"Sun Mar 24 11:15:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57299999999999995,"15min":0.57199999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.57999999999999996,"15min":0.57499999999999996},"min":{"1min":0.41899999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.55100000000000005}]},{"osd":5,"last update":"Sun Mar 24 11:15:42 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.439}]},{"osd":6,"last update":"Sun Mar 24 11:15:02 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.48599999999999999}]},{"osd":7,"last update":"Sun Mar 24 11:15:31 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.45400000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411372,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":6280,"kb_used_data":576,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93738872,"statfs":{"total":95995035648,"available":95988604928,"internally_reserved":0,"allocated":589824,"data_stored":240660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Mar 24 11:15:35 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.375,"15min":0.36499999999999999},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.316},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.38300000000000001,"15min":0.36899999999999999},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.42999999999999999}]},{"osd":1,"last update":"Sun Mar 24 11:15:35 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.374,"15min":0.35999999999999999},"min":{"1min":0.25600000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.40000000000000002,"15min":0.38500000000000001},"min":{"1min":0.27400000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.56799999999999995,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.375}]},{"osd":3,"last update":"Sun Mar 24 11:15:56 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.46300000000000002,"15min":0.46899999999999997},"min":{"1min":0.30399999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.61899999999999999,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.442,"15min":0.44},"min":{"1min":0.29699999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.36199999999999999}]},{"osd":4,"last update":"Sun Mar 24 11:15:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.40100000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.73899999999999999,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.39000000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.55700000000000005}]},{"osd":5,"last update":"Sun Mar 24 11:15:46 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.62,"15min":0.63},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.73599999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.63500000000000001,"15min":0.63700000000000001},"min":{"1min":0.44500000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.81899999999999995,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.72099999999999997}]},{"osd":6,"last update":"Sun Mar 24 11:15:05 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.66600000000000004}]},{"osd":7,"last update":"Sun Mar 24 11:15:30 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.72699999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149735,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5704,"kb_used_data":576,"kb_used_omap":0,"kb_used_meta":5120,"kb_avail":93739448,"statfs":{"total":95995035648,"available":95989194752,"internally_reserved":0,"allocated":589824,"data_stored":240660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5242880},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Mar 24 11:14:57 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.377,"15min":0.376},"min":{"1min":0.25700000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.39200000000000002,"15min":0.39000000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.54000000000000004,"5min":0.629,"15min":0.629},"last":0.88400000000000001}]},{"osd":1,"last update":"Sun Mar 24 11:14:57 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.42099999999999999,"15min":0.42799999999999999},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.46500000000000002,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.41899999999999998,"15min":0.42099999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":1.0449999999999999}]},{"osd":2,"last update":"Sun Mar 24 11:14:57 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47799999999999998,"15min":0.47899999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":2.6909999999999998},{"interface":"front","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.52800000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":2.6819999999999999}]},{"osd":4,"last update":"Sun Mar 24 11:15:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58799999999999997,"15min":0.58399999999999996},"min":{"1min":0.40799999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.52900000000000003}]},{"osd":5,"last update":"Sun Mar 24 11:15:45 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.442,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.78100000000000003,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.59999999999999998,"15min":0.60599999999999998},"min":{"1min":0.42499999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.68000000000000005,"5min":1.117,"15min":1.117},"last":0.72799999999999998}]},{"osd":6,"last update":"Sun Mar 24 11:15:03 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":16.555}]},{"osd":7,"last update":"Sun Mar 24 11:15:31 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":16.567}]}]},{"osd":4,"up_from":31,"seq":133143986211,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5768,"kb_used_data":576,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":93739384,"statfs":{"total":95995035648,"available":95989129216,"internally_reserved":0,"allocated":589824,"data_stored":240660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Mar 24 11:15:23 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51000000000000001,"15min":0.50900000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.67800000000000005,"5min":0.751,"15min":0.751},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.36399999999999999}]},{"osd":1,"last update":"Sun Mar 24 11:15:23 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.51200000000000001,"15min":0.51000000000000001},"min":{"1min":0.32400000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.72299999999999998,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.35699999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69499999999999995,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.51900000000000002}]},{"osd":2,"last update":"Sun Mar 24 11:15:23 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.51100000000000001,"15min":0.50600000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.54500000000000004,"15min":0.54700000000000004},"min":{"1min":0.36699999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.54000000000000004}]},{"osd":3,"last update":"Sun Mar 24 11:15:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.52800000000000002,"15min":0.52300000000000002},"min":{"1min":0.38400000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54900000000000004,"15min":0.55100000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.753,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.44}]},{"osd":5,"last update":"Sun Mar 24 11:15:42 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.372,"15min":0.36199999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.379,"15min":0.36699999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.48799999999999999}]},{"osd":6,"last update":"Sun Mar 24 11:15:02 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.56599999999999995}]},{"osd":7,"last update":"Sun Mar 24 11:15:33 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.55400000000000005}]}]},{"osd":5,"up_from":39,"seq":167503724573,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5636,"kb_used_data":572,"kb_used_omap":0,"kb_used_meta":5056,"kb_avail":93739516,"statfs":{"total":95995035648,"available":95989264384,"internally_reserved":0,"allocated":585728,"data_stored":240395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5177344},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Mar 24 11:15:43 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.53600000000000003,"15min":0.53400000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.496},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.503,"15min":0.496},"min":{"1min":0.35299999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.46600000000000003}]},{"osd":1,"last update":"Sun Mar 24 11:15:43 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.52600000000000002,"15min":0.52200000000000002},"min":{"1min":0.36599999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.47399999999999998}]},{"osd":2,"last update":"Sun Mar 24 11:15:43 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.51200000000000001,"15min":0.50800000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.376,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.45200000000000001}]},{"osd":3,"last update":"Sun Mar 24 11:15:43 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54100000000000004,"15min":0.53800000000000003},"min":{"1min":0.38400000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.52900000000000003,"15min":0.52400000000000002},"min":{"1min":0.38300000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.57799999999999996}]},{"osd":4,"last update":"Sun Mar 24 11:15:43 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.38400000000000001,"15min":0.378},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.40799999999999997,"15min":0.40100000000000002},"min":{"1min":0.27100000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.36599999999999999}]},{"osd":6,"last update":"Sun Mar 24 11:15:02 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.33800000000000002}]},{"osd":7,"last update":"Sun Mar 24 11:15:31 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.439}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-24T11:16:04.062 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph pg dump --format=json 2024-03-24T11:16:04.259 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:03 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:04.631 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T11:16:04.897 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:16:04 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.62 - - [24/Mar/2024:11:16:04] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:16:05.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:04 smithi062 ceph-mon[24799]: from='client.24508 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-24T11:16:05.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:04 smithi062 ceph-mon[24799]: pgmap v41: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:16:05.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:04 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:05.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:04 smithi062 ceph-mon[29666]: from='client.24508 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-24T11:16:05.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:04 smithi062 ceph-mon[29666]: pgmap v41: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:16:05.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:04 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:05.278 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:04 smithi139 ceph-mon[25719]: from='client.24508 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-24T11:16:05.278 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:04 smithi139 ceph-mon[25719]: pgmap v41: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:16:05.278 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:04 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:06.737 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-24T11:16:06.738 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-03-24T11:16:07.071 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:06 smithi062 ceph-mon[24799]: pgmap v42: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:16:07.072 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:06 smithi062 ceph-mon[29666]: pgmap v42: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:16:07.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:06 smithi139 ceph-mon[25719]: pgmap v42: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:16:07.807 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2024-03-24T11:16:05.845964+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1586784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":8,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":749961216,"kb_used":48908,"kb_used_data":6156,"kb_used_omap":0,"kb_used_meta":42688,"kb_avail":749912308,"statfs":{"total":767960285184,"available":767910203392,"internally_reserved":0,"allocated":6303744,"data_stored":3511211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":43712512},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002104"},"pg_stats":[{"pgid":"1.0","version":"60'128","reported_seq":267,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-03-24T11:14:47.951443+0000","last_change":"2024-03-24T11:14:31.858037+0000","last_active":"2024-03-24T11:14:47.951443+0000","last_peered":"2024-03-24T11:14:47.951443+0000","last_clean":"2024-03-24T11:14:47.951443+0000","last_became_active":"2024-03-24T11:14:31.857616+0000","last_became_peered":"2024-03-24T11:14:31.857616+0000","last_unstale":"2024-03-24T11:14:47.951443+0000","last_undegraded":"2024-03-24T11:14:47.951443+0000","last_fullsized":"2024-03-24T11:14:47.951443+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T11:12:32.424867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T11:12:32.424867+0000","last_clean_scrub_stamp":"2024-03-24T11:12:32.424867+0000","objects_scrubbed":0,"log_size":128,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:08:32.528538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1586784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1586784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":55,"seq":236223201301,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6096,"kb_used_data":1096,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":93739056,"statfs":{"total":95995035648,"available":95988793344,"internally_reserved":0,"allocated":1122304,"data_stored":769588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Mar 24 11:15:28 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.41399999999999998}]},{"osd":1,"last update":"Sun Mar 24 11:15:28 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.47299999999999998}]},{"osd":2,"last update":"Sun Mar 24 11:15:28 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.51400000000000001}]},{"osd":3,"last update":"Sun Mar 24 11:15:28 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.495}]},{"osd":4,"last update":"Sun Mar 24 11:15:28 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.52300000000000002}]},{"osd":5,"last update":"Sun Mar 24 11:15:28 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.53700000000000003}]},{"osd":6,"last update":"Sun Mar 24 11:15:28 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.505},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.55800000000000005}]}]},{"osd":6,"up_from":47,"seq":201863462939,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6152,"kb_used_data":1088,"kb_used_omap":0,"kb_used_meta":5056,"kb_avail":93739000,"statfs":{"total":95995035648,"available":95988736000,"internally_reserved":0,"allocated":1114112,"data_stored":769000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5177344},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Mar 24 11:15:03 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.68400000000000005}]},{"osd":1,"last update":"Sun Mar 24 11:15:03 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.65400000000000003}]},{"osd":2,"last update":"Sun Mar 24 11:15:03 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.53500000000000003}]},{"osd":3,"last update":"Sun Mar 24 11:15:03 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.70199999999999996}]},{"osd":4,"last update":"Sun Mar 24 11:15:03 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.377}]},{"osd":5,"last update":"Sun Mar 24 11:15:03 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.34799999999999998}]},{"osd":7,"last update":"Sun Mar 24 11:15:30 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.42599999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":6344,"kb_used_data":576,"kb_used_omap":0,"kb_used_meta":5760,"kb_avail":93738808,"statfs":{"total":95995035648,"available":95988539392,"internally_reserved":0,"allocated":589824,"data_stored":240660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5898240},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Mar 24 11:15:13 2024","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.32900000000000001,"15min":0.31900000000000001},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.33700000000000002,"15min":0.318},"min":{"1min":0.247,"5min":0.215,"15min":0.215},"max":{"1min":0.57099999999999995,"5min":0.57799999999999996,"15min":0.57799999999999996},"last":0.65900000000000003}]},{"osd":2,"last update":"Sun Mar 24 11:15:34 2024","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.371,"15min":0.36599999999999999},"min":{"1min":0.253,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.56899999999999995,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.749},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.434,"15min":0.42099999999999999},"min":{"1min":0.34200000000000003,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.73599999999999999,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.71099999999999997}]},{"osd":3,"last update":"Sun Mar 24 11:15:02 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.44,"15min":0.443},"min":{"1min":0.316,"5min":0.31,"15min":0.31},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.42499999999999999,"15min":0.42899999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"last":0.69099999999999995}]},{"osd":4,"last update":"Sun Mar 24 11:15:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.58699999999999997,"15min":0.59699999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.76800000000000002,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.67600000000000005,"15min":0.70199999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.78100000000000003,"5min":3.4460000000000002,"15min":3.4460000000000002},"last":0.79700000000000004}]},{"osd":5,"last update":"Sun Mar 24 11:15:39 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.55000000000000004,"15min":0.55400000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72099999999999997,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56999999999999995,"15min":0.57499999999999996},"min":{"1min":0.41799999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.81100000000000005}]},{"osd":6,"last update":"Sun Mar 24 11:15:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.78800000000000003}]},{"osd":7,"last update":"Sun Mar 24 11:15:28 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":5.2210000000000001,"5min":5.2210000000000001,"15min":5.2210000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.85099999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738422,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6928,"kb_used_data":1096,"kb_used_omap":0,"kb_used_meta":5824,"kb_avail":93738224,"statfs":{"total":95995035648,"available":95987941376,"internally_reserved":0,"allocated":1122304,"data_stored":769588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5963776},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Mar 24 11:15:13 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.35699999999999998,"15min":0.33700000000000002},"min":{"1min":0.247,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.58699999999999997,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.747},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.377,"15min":0.35599999999999998},"min":{"1min":0.27800000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.89800000000000002}]},{"osd":2,"last update":"Sun Mar 24 11:15:36 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.39100000000000001,"15min":0.378},"min":{"1min":0.28100000000000003,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":1.052},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.39000000000000001,"15min":0.38},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.61199999999999999,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.92000000000000004}]},{"osd":3,"last update":"Sun Mar 24 11:16:02 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.45700000000000002,"15min":0.45100000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":1.093},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.44600000000000001,"15min":0.44800000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.72999999999999998}]},{"osd":4,"last update":"Sun Mar 24 11:15:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57299999999999995,"15min":0.57199999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.57999999999999996,"15min":0.57499999999999996},"min":{"1min":0.41899999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.96099999999999997}]},{"osd":5,"last update":"Sun Mar 24 11:15:42 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.94099999999999995}]},{"osd":6,"last update":"Sun Mar 24 11:16:02 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.626,"15min":0.629},"min":{"1min":0.40600000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91500000000000004,"5min":1.149,"15min":1.149},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.64400000000000002,"15min":0.64600000000000002},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.92000000000000004,"5min":1.085,"15min":1.085},"last":0.878}]},{"osd":7,"last update":"Sun Mar 24 11:15:31 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":1.012},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.82699999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411373,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":6280,"kb_used_data":576,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93738872,"statfs":{"total":95995035648,"available":95988604928,"internally_reserved":0,"allocated":589824,"data_stored":240660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Mar 24 11:15:35 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.375,"15min":0.36499999999999999},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.38300000000000001,"15min":0.36899999999999999},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.48199999999999998}]},{"osd":1,"last update":"Sun Mar 24 11:15:35 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.374,"15min":0.35999999999999999},"min":{"1min":0.25600000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.438},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.40000000000000002,"15min":0.38500000000000001},"min":{"1min":0.27400000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.56799999999999995,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.46800000000000003}]},{"osd":3,"last update":"Sun Mar 24 11:15:56 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.46300000000000002,"15min":0.46899999999999997},"min":{"1min":0.30399999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.61899999999999999,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.442,"15min":0.44},"min":{"1min":0.29699999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.45800000000000002}]},{"osd":4,"last update":"Sun Mar 24 11:15:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.40100000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.73899999999999999,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.504},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.39000000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.51100000000000001}]},{"osd":5,"last update":"Sun Mar 24 11:15:46 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.62,"15min":0.63},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.73599999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.496},{"interface":"front","average":{"1min":0.625,"5min":0.63500000000000001,"15min":0.63700000000000001},"min":{"1min":0.44500000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.81899999999999995,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.66700000000000004}]},{"osd":6,"last update":"Sun Mar 24 11:15:05 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.57099999999999995}]},{"osd":7,"last update":"Sun Mar 24 11:15:30 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.76100000000000001}]}]},{"osd":3,"up_from":26,"seq":111669149736,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5704,"kb_used_data":576,"kb_used_omap":0,"kb_used_meta":5120,"kb_avail":93739448,"statfs":{"total":95995035648,"available":95989194752,"internally_reserved":0,"allocated":589824,"data_stored":240660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5242880},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Mar 24 11:16:02 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.376,"15min":0.376},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.52000000000000002,"5min":0.629,"15min":0.629},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.39400000000000002,"15min":0.39100000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.34200000000000003}]},{"osd":1,"last update":"Sun Mar 24 11:16:02 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41699999999999998,"15min":0.42699999999999999},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41999999999999998},"min":{"1min":0.26200000000000001,"5min":0.248,"15min":0.248},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.42399999999999999}]},{"osd":2,"last update":"Sun Mar 24 11:16:02 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.49399999999999999,"15min":0.48399999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":2.6909999999999998,"5min":2.6909999999999998,"15min":2.6909999999999998},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.46600000000000003,"15min":0.45000000000000001},"min":{"1min":0.32400000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":2.6819999999999999,"5min":2.6819999999999999,"15min":2.6819999999999999},"last":0.48099999999999998}]},{"osd":4,"last update":"Sun Mar 24 11:15:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58799999999999997,"15min":0.58399999999999996},"min":{"1min":0.40799999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.58599999999999997}]},{"osd":5,"last update":"Sun Mar 24 11:15:45 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.442,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.78100000000000003,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.59999999999999998,"15min":0.60599999999999998},"min":{"1min":0.42499999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.68000000000000005,"5min":1.117,"15min":1.117},"last":0.505}]},{"osd":6,"last update":"Sun Mar 24 11:15:03 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.53800000000000003}]},{"osd":7,"last update":"Sun Mar 24 11:15:31 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.60899999999999999}]}]},{"osd":4,"up_from":31,"seq":133143986212,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5768,"kb_used_data":576,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":93739384,"statfs":{"total":95995035648,"available":95989129216,"internally_reserved":0,"allocated":589824,"data_stored":240660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Mar 24 11:15:23 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51000000000000001,"15min":0.50900000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.67800000000000005,"5min":0.751,"15min":0.751},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.503}]},{"osd":1,"last update":"Sun Mar 24 11:15:23 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.51200000000000001,"15min":0.51000000000000001},"min":{"1min":0.32400000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.72299999999999998,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.44},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.35699999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69499999999999995,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.39600000000000002}]},{"osd":2,"last update":"Sun Mar 24 11:15:23 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.51100000000000001,"15min":0.50600000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.54500000000000004,"15min":0.54700000000000004},"min":{"1min":0.36699999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.43099999999999999}]},{"osd":3,"last update":"Sun Mar 24 11:15:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.52800000000000002,"15min":0.52300000000000002},"min":{"1min":0.38400000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54900000000000004,"15min":0.55100000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.753,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.38700000000000001}]},{"osd":5,"last update":"Sun Mar 24 11:15:42 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.372,"15min":0.36199999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.379,"15min":0.36699999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.32600000000000001}]},{"osd":6,"last update":"Sun Mar 24 11:15:02 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.41999999999999998}]},{"osd":7,"last update":"Sun Mar 24 11:15:33 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.41399999999999998}]}]},{"osd":5,"up_from":39,"seq":167503724574,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5636,"kb_used_data":572,"kb_used_omap":0,"kb_used_meta":5056,"kb_avail":93739516,"statfs":{"total":95995035648,"available":95989264384,"internally_reserved":0,"allocated":585728,"data_stored":240395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5177344},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Mar 24 11:15:43 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.53600000000000003,"15min":0.53400000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.374},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.503,"15min":0.496},"min":{"1min":0.35299999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.54700000000000004}]},{"osd":1,"last update":"Sun Mar 24 11:15:43 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.52600000000000002,"15min":0.52200000000000002},"min":{"1min":0.36599999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.42499999999999999}]},{"osd":2,"last update":"Sun Mar 24 11:15:43 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.51200000000000001,"15min":0.50800000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.376,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.47599999999999998}]},{"osd":3,"last update":"Sun Mar 24 11:15:43 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54100000000000004,"15min":0.53800000000000003},"min":{"1min":0.38400000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.52900000000000003,"15min":0.52400000000000002},"min":{"1min":0.38300000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.45500000000000002}]},{"osd":4,"last update":"Sun Mar 24 11:15:43 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.38400000000000001,"15min":0.378},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.31},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.40799999999999997,"15min":0.40100000000000002},"min":{"1min":0.27100000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.39500000000000002}]},{"osd":6,"last update":"Sun Mar 24 11:15:02 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.437},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.45000000000000001}]},{"osd":7,"last update":"Sun Mar 24 11:15:31 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.497}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-24T11:16:07.809 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-24T11:16:07.809 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-24T11:16:07.809 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-24T11:16:07.810 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- ceph health --format=json 2024-03-24T11:16:08.068 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T11:16:08.191 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:07 smithi062 ceph-mon[29666]: from='client.14607 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-24T11:16:08.191 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:07 smithi062 ceph-mon[24799]: from='client.14607 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-24T11:16:08.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:07 smithi139 ceph-mon[25719]: from='client.14607 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-24T11:16:09.081 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:08 smithi062 ceph-mon[24799]: pgmap v43: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:16:09.081 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:08 smithi062 ceph-mon[29666]: pgmap v43: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:16:09.082 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:16:08 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:16:08] "GET /metrics HTTP/1.1" 200 191491 "" "Prometheus/2.33.4" 2024-03-24T11:16:09.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:08 smithi139 ceph-mon[25719]: pgmap v43: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:16:09.352 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:16:09 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[49311]: level=info ts=2024-03-24T11:16:09.089Z caller=cluster.go:688 component=cluster msg="gossip settled; proceeding" elapsed=10.002557793s 2024-03-24T11:16:09.826 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-03-24T11:16:09.826 INFO:teuthology.orchestra.run.smithi062.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-24T11:16:10.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:09 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/4133526306' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-24T11:16:10.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:09 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/4133526306' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-24T11:16:10.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:09 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/4133526306' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-24T11:16:10.557 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-24T11:16:10.557 INFO:tasks.cephadm:Setup complete, yielding 2024-03-24T11:16:10.558 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-24T11:16:10.568 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi062.front.sepia.ceph.com 2024-03-24T11:16:10.568 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest false --force' 2024-03-24T11:16:10.858 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:16:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:16:10] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-03-24T11:16:11.185 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:10 smithi139 ceph-mon[25719]: pgmap v44: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:16:11.187 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:10 smithi062 ceph-mon[24799]: pgmap v44: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:16:11.187 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:10 smithi062 ceph-mon[29666]: pgmap v44: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:16:11.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:11 smithi062 ceph-mon[24799]: pgmap v45: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:16:11.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:11 smithi062 ceph-mon[29666]: pgmap v45: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:16:12.198 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:11 smithi139 ceph-mon[25719]: pgmap v45: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:16:12.743 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="copy alert_rule_tag v1 to v2" 2024-03-24T11:16:12.743 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="drop table alert_rule_tag_v1" 2024-03-24T11:16:12.995 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="create alert_notification table v1" 2024-03-24T11:16:12.995 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Add column is_default" 2024-03-24T11:16:12.995 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Add column frequency" 2024-03-24T11:16:12.995 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Add column send_reminder" 2024-03-24T11:16:12.995 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Add column disable_resolve_message" 2024-03-24T11:16:12.995 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert_notification org_id & name" 2024-03-24T11:16:12.995 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Update alert table charset" 2024-03-24T11:16:12.995 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Update alert_notification table charset" 2024-03-24T11:16:12.995 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="create notification_journal table v1" 2024-03-24T11:16:12.995 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add index notification_journal org_id & alert_id & notifier_id" 2024-03-24T11:16:12.996 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="drop alert_notification_journal" 2024-03-24T11:16:12.996 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="create alert_notification_state table v1" 2024-03-24T11:16:12.996 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-03-24T11:16:12.996 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Add for to alert table" 2024-03-24T11:16:12.996 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Add column uid in alert_notification" 2024-03-24T11:16:12.996 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Update uid column values in alert_notification" 2024-03-24T11:16:12.996 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index alert_notification_org_id_uid" 2024-03-24T11:16:12.996 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Remove unique index org_id_name" 2024-03-24T11:16:12.996 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Add column secure_settings in alert_notification" 2024-03-24T11:16:12.997 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="alter alert.settings to mediumtext" 2024-03-24T11:16:12.997 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Add non-unique index alert_notification_state_alert_id" 2024-03-24T11:16:12.998 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Add non-unique index alert_rule_tag_alert_id" 2024-03-24T11:16:12.998 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old annotation table v4" 2024-03-24T11:16:12.998 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="create annotation table v5" 2024-03-24T11:16:12.998 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 0 v3" 2024-03-24T11:16:12.998 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 1 v3" 2024-03-24T11:16:12.998 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 2 v3" 2024-03-24T11:16:12.998 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 3 v3" 2024-03-24T11:16:12.998 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 4 v3" 2024-03-24T11:16:12.998 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Update annotation table charset" 2024-03-24T11:16:12.998 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Add column region_id to annotation table" 2024-03-24T11:16:12.998 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Drop category_id index" 2024-03-24T11:16:12.998 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Add column tags to annotation table" 2024-03-24T11:16:12.998 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Create annotation_tag table v2" 2024-03-24T11:16:12.998 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index annotation_tag.annotation_id_tag_id" 2024-03-24T11:16:13.003 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-03-24T11:16:13.003 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-03-24T11:16:13.003 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Create annotation_tag table v3" 2024-03-24T11:16:13.003 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-03-24T11:16:13.003 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="copy annotation_tag v2 to v3" 2024-03-24T11:16:13.003 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="drop table annotation_tag_v2" 2024-03-24T11:16:13.003 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Update alert annotations and set TEXT to empty" 2024-03-24T11:16:13.003 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Add created time to annotation table" 2024-03-24T11:16:13.003 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Add updated time to annotation table" 2024-03-24T11:16:13.004 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for created in annotation table" 2024-03-24T11:16:13.004 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for updated in annotation table" 2024-03-24T11:16:13.004 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Convert existing annotations from seconds to milliseconds" 2024-03-24T11:16:13.004 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Add epoch_end column" 2024-03-24T11:16:13.004 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for epoch_end" 2024-03-24T11:16:13.004 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Make epoch_end the same as epoch" 2024-03-24T11:16:13.004 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Move region to single row" 2024-03-24T11:16:13.004 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Remove index org_id_epoch from annotation table" 2024-03-24T11:16:13.004 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-03-24T11:16:13.004 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-03-24T11:16:13.004 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for org_id_epoch_end_epoch on annotation table" 2024-03-24T11:16:13.004 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Remove index org_id_epoch_epoch_end from annotation table" 2024-03-24T11:16:13.005 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for alert_id on annotation table" 2024-03-24T11:16:13.005 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="create test_data table" 2024-03-24T11:16:13.005 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_version table v1" 2024-03-24T11:16:13.005 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_version.dashboard_id" 2024-03-24T11:16:13.005 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-03-24T11:16:13.005 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Set dashboard version to 1 where 0" 2024-03-24T11:16:13.005 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="save existing dashboard data in dashboard_version table v1" 2024-03-24T11:16:13.005 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="alter dashboard_version.data to mediumtext v1" 2024-03-24T11:16:13.005 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="create team table" 2024-03-24T11:16:13.005 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add index team.org_id" 2024-03-24T11:16:13.005 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index team_org_id_name" 2024-03-24T11:16:13.005 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="create team member table" 2024-03-24T11:16:13.005 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add index team_member.org_id" 2024-03-24T11:16:13.006 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index team_member_org_id_team_id_user_id" 2024-03-24T11:16:13.006 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add index team_member.team_id" 2024-03-24T11:16:13.006 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Add column email to team table" 2024-03-24T11:16:13.006 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Add column external to team_member table" 2024-03-24T11:16:13.006 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Add column permission to team_member table" 2024-03-24T11:16:13.006 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard acl table" 2024-03-24T11:16:13.006 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_acl_dashboard_id" 2024-03-24T11:16:13.006 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index dashboard_acl_dashboard_id_user_id" 2024-03-24T11:16:13.006 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index dashboard_acl_dashboard_id_team_id" 2024-03-24T11:16:13.006 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_acl_user_id" 2024-03-24T11:16:13.006 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_acl_team_id" 2024-03-24T11:16:13.006 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_acl_org_id_role" 2024-03-24T11:16:13.006 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_permission" 2024-03-24T11:16:13.006 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="save default acl rules in dashboard_acl table" 2024-03-24T11:16:13.006 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="delete acl rules for deleted dashboards and folders" 2024-03-24T11:16:13.007 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="create tag table" 2024-03-24T11:16:13.007 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add index tag.key_value" 2024-03-24T11:16:13.007 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="create login attempt table" 2024-03-24T11:16:13.007 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add index login_attempt.username" 2024-03-24T11:16:13.007 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_login_attempt_username - v1" 2024-03-24T11:16:13.007 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-03-24T11:16:13.007 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="create login_attempt v2" 2024-03-24T11:16:13.007 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_login_attempt_username - v2" 2024-03-24T11:16:13.007 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="copy login_attempt v1 to v2" 2024-03-24T11:16:13.007 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="drop login_attempt_tmp_qwerty" 2024-03-24T11:16:13.007 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="create user auth table" 2024-03-24T11:16:13.007 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-03-24T11:16:13.007 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="alter user_auth.auth_id to length 190" 2024-03-24T11:16:13.008 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Add OAuth access token to user_auth" 2024-03-24T11:16:13.008 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Add OAuth refresh token to user_auth" 2024-03-24T11:16:13.008 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Add OAuth token type to user_auth" 2024-03-24T11:16:13.008 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Add OAuth expiry to user_auth" 2024-03-24T11:16:13.008 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Add index to user_id column in user_auth" 2024-03-24T11:16:13.008 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="create server_lock table" 2024-03-24T11:16:13.008 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add index server_lock.operation_uid" 2024-03-24T11:16:13.008 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="create user auth token table" 2024-03-24T11:16:13.008 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index user_auth_token.auth_token" 2024-03-24T11:16:13.008 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index user_auth_token.prev_auth_token" 2024-03-24T11:16:13.008 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add index user_auth_token.user_id" 2024-03-24T11:16:13.008 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Add revoked_at to the user auth token" 2024-03-24T11:16:13.008 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="create cache_data table" 2024-03-24T11:16:13.008 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index cache_data.cache_key" 2024-03-24T11:16:13.009 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="create short_url table v1" 2024-03-24T11:16:13.009 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add index short_url.org_id-uid" 2024-03-24T11:16:13.009 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="delete alert_definition table" 2024-03-24T11:16:13.009 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="recreate alert_definition table" 2024-03-24T11:16:13.009 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_definition on org_id and title columns" 2024-03-24T11:16:13.009 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_definition on org_id and uid columns" 2024-03-24T11:16:13.009 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="alter alert_definition table data column to mediumtext in mysql" 2024-03-24T11:16:13.009 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="drop index in alert_definition on org_id and title columns" 2024-03-24T11:16:13.009 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="drop index in alert_definition on org_id and uid columns" 2024-03-24T11:16:13.009 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index in alert_definition on org_id and title columns" 2024-03-24T11:16:13.009 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index in alert_definition on org_id and uid columns" 2024-03-24T11:16:13.009 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Add column paused in alert_definition" 2024-03-24T11:16:13.009 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="drop alert_definition table" 2024-03-24T11:16:13.009 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="delete alert_definition_version table" 2024-03-24T11:16:13.010 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="recreate alert_definition_version table" 2024-03-24T11:16:13.010 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-03-24T11:16:13.010 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-03-24T11:16:13.010 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="alter alert_definition_version table data column to mediumtext in mysql" 2024-03-24T11:16:13.010 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="drop alert_definition_version table" 2024-03-24T11:16:13.010 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="create alert_instance table" 2024-03-24T11:16:13.010 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-03-24T11:16:13.010 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_instance table on def_org_id, current_state columns" 2024-03-24T11:16:13.010 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add column current_state_end to alert_instance" 2024-03-24T11:16:13.010 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-03-24T11:16:13.010 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="remove index def_org_id, current_state on alert_instance" 2024-03-24T11:16:13.010 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="rename def_org_id to rule_org_id in alert_instance" 2024-03-24T11:16:13.010 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="rename def_uid to rule_uid in alert_instance" 2024-03-24T11:16:13.010 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-03-24T11:16:13.011 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add index rule_org_id, current_state on alert_instance" 2024-03-24T11:16:13.011 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="create alert_rule table" 2024-03-24T11:16:13.011 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule on org_id and title columns" 2024-03-24T11:16:13.011 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule on org_id and uid columns" 2024-03-24T11:16:13.011 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-03-24T11:16:13.011 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="alter alert_rule table data column to mediumtext in mysql" 2024-03-24T11:16:13.011 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add column for to alert_rule" 2024-03-24T11:16:13.011 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add column annotations to alert_rule" 2024-03-24T11:16:13.011 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add column labels to alert_rule" 2024-03-24T11:16:13.011 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="remove unique index from alert_rule on org_id, title columns" 2024-03-24T11:16:13.011 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-03-24T11:16:13.011 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add dashboard_uid column to alert_rule" 2024-03-24T11:16:13.011 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add panel_id column to alert_rule" 2024-03-24T11:16:13.011 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-03-24T11:16:13.011 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="create alert_rule_version table" 2024-03-24T11:16:13.012 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-03-24T11:16:13.012 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-03-24T11:16:13.012 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="alter alert_rule_version table data column to mediumtext in mysql" 2024-03-24T11:16:13.012 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add column for to alert_rule_version" 2024-03-24T11:16:13.012 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add column annotations to alert_rule_version" 2024-03-24T11:16:13.012 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add column labels to alert_rule_version" 2024-03-24T11:16:13.012 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id=create_alert_configuration_table 2024-03-24T11:16:13.012 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="Add column default in alert_configuration" 2024-03-24T11:16:13.013 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-03-24T11:16:13.013 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add column org_id in alert_configuration" 2024-03-24T11:16:13.013 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_configuration table on org_id column" 2024-03-24T11:16:13.013 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id=create_ngalert_configuration_table 2024-03-24T11:16:13.013 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add index in ngalert_configuration on org_id column" 2024-03-24T11:16:13.013 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="clear migration entry \"remove unified alerting data\"" 2024-03-24T11:16:13.013 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="move dashboard alerts to unified alerting" 2024-03-24T11:16:13.013 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="create library_element table v1" 2024-03-24T11:16:13.013 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add index library_element org_id-folder_id-name-kind" 2024-03-24T11:16:13.013 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="create library_element_connection table v1" 2024-03-24T11:16:13.013 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add index library_element_connection element_id-kind-connection_id" 2024-03-24T11:16:13.013 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index library_element org_id_uid" 2024-03-24T11:16:13.013 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="clone move dashboard alerts to unified alerting" 2024-03-24T11:16:13.013 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="create data_keys table" 2024-03-24T11:16:13.014 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="create kv_store table v1" 2024-03-24T11:16:13.014 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add index kv_store.org_id-namespace-key" 2024-03-24T11:16:13.014 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="update dashboard_uid and panel_id from existing annotations" 2024-03-24T11:16:13.014 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="create permission table" 2024-03-24T11:16:13.014 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index permission.role_id" 2024-03-24T11:16:13.014 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index role_id_action_scope" 2024-03-24T11:16:13.358 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:12 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:12+0000 lvl=info msg="Executing migration" logger=migrator id="create role table" 2024-03-24T11:16:13.358 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:13+0000 lvl=info msg="Executing migration" logger=migrator id="add column display_name" 2024-03-24T11:16:13.358 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:13+0000 lvl=info msg="Executing migration" logger=migrator id="add column group_name" 2024-03-24T11:16:13.359 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:13+0000 lvl=info msg="Executing migration" logger=migrator id="add index role.org_id" 2024-03-24T11:16:13.359 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:13+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index role_org_id_name" 2024-03-24T11:16:13.359 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:13+0000 lvl=info msg="Executing migration" logger=migrator id="add index role_org_id_uid" 2024-03-24T11:16:13.359 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:13+0000 lvl=info msg="Executing migration" logger=migrator id="create team role table" 2024-03-24T11:16:13.359 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:13+0000 lvl=info msg="Executing migration" logger=migrator id="add index team_role.org_id" 2024-03-24T11:16:13.359 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:13+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index team_role_org_id_team_id_role_id" 2024-03-24T11:16:13.359 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:13+0000 lvl=info msg="Executing migration" logger=migrator id="add index team_role.team_id" 2024-03-24T11:16:13.359 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:13+0000 lvl=info msg="Executing migration" logger=migrator id="create user role table" 2024-03-24T11:16:13.359 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:13+0000 lvl=info msg="Executing migration" logger=migrator id="add index user_role.org_id" 2024-03-24T11:16:13.359 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:13+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index user_role_org_id_user_id_role_id" 2024-03-24T11:16:13.359 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:13+0000 lvl=info msg="Executing migration" logger=migrator id="add index user_role.user_id" 2024-03-24T11:16:13.359 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:13+0000 lvl=info msg="Executing migration" logger=migrator id="create builtin role table" 2024-03-24T11:16:13.359 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:13+0000 lvl=info msg="Executing migration" logger=migrator id="add index builtin_role.role_id" 2024-03-24T11:16:13.359 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:13+0000 lvl=info msg="Executing migration" logger=migrator id="add index builtin_role.name" 2024-03-24T11:16:13.360 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:13+0000 lvl=info msg="Executing migration" logger=migrator id="Add column org_id to builtin_role table" 2024-03-24T11:16:13.360 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:13+0000 lvl=info msg="Executing migration" logger=migrator id="add index builtin_role.org_id" 2024-03-24T11:16:13.360 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:13+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index builtin_role_org_id_role_id_role" 2024-03-24T11:16:13.360 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:13+0000 lvl=info msg="Executing migration" logger=migrator id="Remove unique index role_org_id_uid" 2024-03-24T11:16:13.360 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:13+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index role.uid" 2024-03-24T11:16:13.360 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:13+0000 lvl=info msg="Executing migration" logger=migrator id="create seed assignment table" 2024-03-24T11:16:13.360 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:13+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index builtin_role_role_name" 2024-03-24T11:16:13.360 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:13+0000 lvl=info msg="migrations completed" logger=migrator performed=381 skipped=0 duration=496.814538ms 2024-03-24T11:16:13.360 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:13+0000 lvl=info msg="Created default organization" logger=sqlstore 2024-03-24T11:16:13.360 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:13+0000 lvl=info msg="Initialising plugins" logger=plugin.manager 2024-03-24T11:16:13.360 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:13+0000 lvl=info msg="Plugin registered" logger=plugin.manager pluginId=input 2024-03-24T11:16:13.360 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:13+0000 lvl=info msg="Plugin registered" logger=plugin.manager pluginId=vonage-status-panel 2024-03-24T11:16:13.360 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:13+0000 lvl=info msg="Plugin registered" logger=plugin.manager pluginId=grafana-piechart-panel 2024-03-24T11:16:13.360 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:13+0000 lvl=info msg="Live Push Gateway initialization" logger=live.push_http 2024-03-24T11:16:13.360 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:13+0000 lvl=warn msg="[Deprecated] the datasource provisioning config is outdated. please upgrade" logger=provisioning.datasources filename=/etc/grafana/provisioning/datasources/ceph-dashboard.yml 2024-03-24T11:16:13.361 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:13+0000 lvl=info msg="inserting datasource from configuration " logger=provisioning.datasources name=Dashboard1 uid=P43CA22E17D0F9596 2024-03-24T11:16:13.361 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:13+0000 lvl=info msg="warming cache for startup" logger=ngalert 2024-03-24T11:16:13.361 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:13+0000 lvl=info msg="HTTP Server Listen" logger=http.server address=[::]:3000 protocol=https subUrl= socket= 2024-03-24T11:16:13.361 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:16:13+0000 lvl=info msg="starting MultiOrg Alertmanager" logger=ngalert.multiorg.alertmanager 2024-03-24T11:16:13.429 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.62 - - [24/Mar/2024:11:16:13] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:16:13.702 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:13.703 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:16:13.703 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:16:13.703 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:16:13.703 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:13.703 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:16:13.703 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-24T11:16:13.703 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[24799]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-24T11:16:13.703 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://172.21.15.139:3000"}]: dispatch 2024-03-24T11:16:13.703 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[24799]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://172.21.15.139:3000"}]: dispatch 2024-03-24T11:16:13.703 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:13.703 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:16:13.703 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:16:13.703 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:16:13.703 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:13.704 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:16:13.704 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:16:13.704 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:16:13.704 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:16:13.704 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:13.704 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:16:13.704 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:13.704 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:13.704 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:13.704 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:16:13.704 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:16:13.704 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:16:13.704 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:13.704 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/2837926810' entity='client.admin' 2024-03-24T11:16:13.705 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:13.705 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:16:13.705 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:16:13.705 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:16:13.705 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:13.705 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:16:13.705 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-24T11:16:13.705 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[29666]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-24T11:16:13.705 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://172.21.15.139:3000"}]: dispatch 2024-03-24T11:16:13.705 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[29666]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://172.21.15.139:3000"}]: dispatch 2024-03-24T11:16:13.705 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:13.705 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:16:13.705 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:16:13.705 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:16:13.706 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:13.706 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:16:13.706 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:16:13.706 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:16:13.706 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:16:13.706 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:13.706 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:16:13.706 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:13.706 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:13.706 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:13.706 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:16:13.706 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:16:13.706 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:16:13.706 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:13.707 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/2837926810' entity='client.admin' 2024-03-24T11:16:13.803 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:13.803 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:16:13.803 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:16:13.803 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:16:13.804 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:13.804 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:16:13.804 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-24T11:16:13.804 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-mon[25719]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-24T11:16:13.804 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://172.21.15.139:3000"}]: dispatch 2024-03-24T11:16:13.804 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-mon[25719]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://172.21.15.139:3000"}]: dispatch 2024-03-24T11:16:13.804 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:13.804 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:16:13.804 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:16:13.804 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:16:13.804 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:13.804 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:16:13.804 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:16:13.804 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:16:13.804 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:16:13.805 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:13.805 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:16:13.805 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:13.805 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:13.805 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:13.805 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:16:13.805 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:16:13.805 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:16:13.805 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:13.805 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:13 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/2837926810' entity='client.admin' 2024-03-24T11:16:13.976 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:16:13 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.139 - - [24/Mar/2024:11:16:13] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:16:15.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:14 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:16:15.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:14 smithi139 ceph-mon[25719]: Reconfiguring alertmanager.a (dependencies changed)... 2024-03-24T11:16:15.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:14 smithi139 ceph-mon[25719]: Reconfiguring daemon alertmanager.a on smithi062 2024-03-24T11:16:15.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:14 smithi139 ceph-mon[25719]: pgmap v46: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:16:15.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:14 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:15.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:14 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:15.170 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:14 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:16:15.170 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:14 smithi062 ceph-mon[29666]: Reconfiguring alertmanager.a (dependencies changed)... 2024-03-24T11:16:15.170 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:14 smithi062 ceph-mon[29666]: Reconfiguring daemon alertmanager.a on smithi062 2024-03-24T11:16:15.170 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:14 smithi062 ceph-mon[29666]: pgmap v46: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:16:15.170 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:14 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:15.170 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:14 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:15.170 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:14 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:16:15.170 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:14 smithi062 ceph-mon[24799]: Reconfiguring alertmanager.a (dependencies changed)... 2024-03-24T11:16:15.170 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:14 smithi062 ceph-mon[24799]: Reconfiguring daemon alertmanager.a on smithi062 2024-03-24T11:16:15.170 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:14 smithi062 ceph-mon[24799]: pgmap v46: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:16:15.170 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:14 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:15.170 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:14 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:16.143 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-24T11:16:16.154 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi062.front.sepia.ceph.com 2024-03-24T11:16:16.154 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'radosgw-admin realm create --rgw-realm=r --default' 2024-03-24T11:16:17.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:16 smithi062 ceph-mon[24799]: pgmap v47: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:16:17.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:16 smithi062 ceph-mon[29666]: pgmap v47: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:16:17.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:16 smithi139 ceph-mon[25719]: pgmap v47: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:16:18.226 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:16:18 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.139 - - [24/Mar/2024:11:16:18] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:16:18.726 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:16:18 smithi062 systemd[1]: Stopping Ceph alertmanager.a for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:16:19.065 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:16:18 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:16:18] "GET /metrics HTTP/1.1" 200 191497 "" "Prometheus/2.33.4" 2024-03-24T11:16:19.066 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:18 smithi062 ceph-mon[24799]: pgmap v48: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:16:19.066 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:18 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:19.066 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:16:18 smithi062 bash[50786]: Error: no container with name or ID "ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager.a" found: no such container 2024-03-24T11:16:19.066 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:16:18 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[49311]: level=info ts=2024-03-24T11:16:18.866Z caller=main.go:557 msg="Received SIGTERM, exiting gracefully..." 2024-03-24T11:16:19.066 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:18 smithi062 ceph-mon[29666]: pgmap v48: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:16:19.067 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:18 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:19.282 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:18 smithi139 ceph-mon[25719]: pgmap v48: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:16:19.282 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:18 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:19.389 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:16:19 smithi062 podman[50799]: 2024-03-24 11:16:19.066798526 +0000 UTC m=+0.314424045 container died 69b54ef5350762804269ebc2f0d9acd593b44efd48a9ec7ed2e33338af7512c6 (image=quay.io/prometheus/alertmanager:v0.23.0, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-24T11:16:19.389 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:16:19 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.139 - - [24/Mar/2024:11:16:19] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:16:19.727 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:16:19 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.139 - - [24/Mar/2024:11:16:19] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:16:19.727 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:16:19 smithi062 podman[50799]: 2024-03-24 11:16:19.48526838 +0000 UTC m=+0.732893892 container cleanup 69b54ef5350762804269ebc2f0d9acd593b44efd48a9ec7ed2e33338af7512c6 (image=quay.io/prometheus/alertmanager:v0.23.0, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-24T11:16:19.727 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:16:19 smithi062 bash[50799]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a 2024-03-24T11:16:19.728 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:16:19 smithi062 bash[50899]: Error: no container with name or ID "ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager.a" found: no such container 2024-03-24T11:16:20.020 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:16:19 smithi062 podman[50815]: 2024-03-24 11:16:19.73626902 +0000 UTC m=+0.843975135 container remove 69b54ef5350762804269ebc2f0d9acd593b44efd48a9ec7ed2e33338af7512c6 (image=quay.io/prometheus/alertmanager:v0.23.0, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-24T11:16:20.021 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:16:19 smithi062 podman[50815]: 2024-03-24 11:16:19.820225357 +0000 UTC m=+0.927931434 volume remove 2e0075e627bc6969d7b8c067893aa716bc55f47f7b2765814d680ff1b0af0be5 2024-03-24T11:16:20.021 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:16:19 smithi062 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@alertmanager.a.service: Deactivated successfully. 2024-03-24T11:16:20.021 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:16:19 smithi062 systemd[1]: Stopped Ceph alertmanager.a for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:16:20.021 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:16:19 smithi062 systemd[1]: Starting Ceph alertmanager.a for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:16:20.280 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:16:20 smithi062 podman[50964]: 2024-03-24 11:16:20.021147815 +0000 UTC m=+0.087626991 volume create ad92132d55dcfdd6dcbe63948733f278052b0a0e98593a4217d2266f5ff2047f 2024-03-24T11:16:20.280 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:16:20 smithi062 podman[50964]: 2024-03-24 11:16:19.951907526 +0000 UTC m=+0.018386703 image pull ba2b418f427c0636d654de8757e830c80168e76482bcc46bb2138e569d6c91d4 quay.io/prometheus/alertmanager:v0.23.0 2024-03-24T11:16:20.280 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:16:20 smithi062 podman[50964]: 2024-03-24 11:16:20.121688203 +0000 UTC m=+0.188167379 container create 168b140d64163b79e7ff94ca02791ed073bca0d30319bde0244b1cc69a88d845 (image=quay.io/prometheus/alertmanager:v0.23.0, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-24T11:16:20.577 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:20 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:20.577 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:20 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:20.577 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:20 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:20.577 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:20 smithi139 ceph-mon[25719]: pgmap v49: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:16:20.577 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:20 smithi139 ceph-mon[25719]: osdmap e61: 8 total, 8 up, 8 in 2024-03-24T11:16:20.577 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:20 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/1986498652' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-03-24T11:16:20.577 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:20 smithi139 ceph-mon[25719]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-03-24T11:16:20.650 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:20 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:20.650 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:20 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:20.650 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:20 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:20.650 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:20 smithi062 ceph-mon[24799]: pgmap v49: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:16:20.650 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:20 smithi062 ceph-mon[24799]: osdmap e61: 8 total, 8 up, 8 in 2024-03-24T11:16:20.651 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:20 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/1986498652' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-03-24T11:16:20.651 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:20 smithi062 ceph-mon[24799]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-03-24T11:16:20.651 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:20 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:20.651 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:20 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:20.651 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:20 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:20.651 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:20 smithi062 ceph-mon[29666]: pgmap v49: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:16:20.651 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:20 smithi062 ceph-mon[29666]: osdmap e61: 8 total, 8 up, 8 in 2024-03-24T11:16:20.651 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:20 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/1986498652' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-03-24T11:16:20.651 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:20 smithi062 ceph-mon[29666]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-03-24T11:16:20.652 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:16:20 smithi062 podman[50964]: 2024-03-24 11:16:20.399433845 +0000 UTC m=+0.465913036 container init 168b140d64163b79e7ff94ca02791ed073bca0d30319bde0244b1cc69a88d845 (image=quay.io/prometheus/alertmanager:v0.23.0, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-24T11:16:20.652 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:16:20 smithi062 podman[50964]: 2024-03-24 11:16:20.402062275 +0000 UTC m=+0.468541459 container start 168b140d64163b79e7ff94ca02791ed073bca0d30319bde0244b1cc69a88d845 (image=quay.io/prometheus/alertmanager:v0.23.0, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-24T11:16:20.652 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:16:20 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=info ts=2024-03-24T11:16:20.411Z caller=main.go:225 msg="Starting Alertmanager" version="(version=0.23.0, branch=HEAD, revision=61046b17771a57cfd4c4a51be370ab930a4d7d54)" 2024-03-24T11:16:20.652 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:16:20 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=info ts=2024-03-24T11:16:20.411Z caller=main.go:226 build_context="(go=go1.16.7, user=root@e21a959be8d2, date=20210825-10:48:55)" 2024-03-24T11:16:20.652 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:16:20 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=info ts=2024-03-24T11:16:20.412Z caller=cluster.go:184 component=cluster msg="setting advertise address explicitly" addr=172.21.15.62 port=9094 2024-03-24T11:16:20.652 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:16:20 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=info ts=2024-03-24T11:16:20.413Z caller=cluster.go:671 component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-03-24T11:16:20.652 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:16:20 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=info ts=2024-03-24T11:16:20.439Z caller=coordinator.go:113 component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-03-24T11:16:20.652 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:16:20 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=info ts=2024-03-24T11:16:20.439Z caller=coordinator.go:126 component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-03-24T11:16:20.652 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:16:20 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=info ts=2024-03-24T11:16:20.442Z caller=main.go:518 msg=Listening address=:9093 2024-03-24T11:16:20.652 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:16:20 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=info ts=2024-03-24T11:16:20.442Z caller=tls_config.go:191 msg="TLS is disabled." http2=false 2024-03-24T11:16:20.652 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:16:20 smithi062 bash[50964]: 168b140d64163b79e7ff94ca02791ed073bca0d30319bde0244b1cc69a88d845 2024-03-24T11:16:20.653 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:16:20 smithi062 systemd[1]: Started Ceph alertmanager.a for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:16:20.858 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:16:20 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:16:20] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-03-24T11:16:20.958 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:16:20.958 INFO:teuthology.orchestra.run.smithi062.stdout: "id": "937f0dc2-2b1d-4d64-b23a-bf9f2400aa57", 2024-03-24T11:16:20.958 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "r", 2024-03-24T11:16:20.958 INFO:teuthology.orchestra.run.smithi062.stdout: "current_period": "f1091927-c182-4e18-b336-f2dc08906f60", 2024-03-24T11:16:20.958 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 1 2024-03-24T11:16:20.959 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:16:21.608 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:21 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:21.609 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:21 smithi062 ceph-mon[29666]: Reconfiguring prometheus.a (dependencies changed)... 2024-03-24T11:16:21.609 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:21 smithi062 ceph-mon[29666]: Reconfiguring daemon prometheus.a on smithi139 2024-03-24T11:16:21.609 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:21 smithi062 ceph-mon[29666]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-03-24T11:16:21.609 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:21 smithi062 ceph-mon[29666]: osdmap e62: 8 total, 8 up, 8 in 2024-03-24T11:16:21.609 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:21 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:21.609 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:21 smithi062 ceph-mon[24799]: Reconfiguring prometheus.a (dependencies changed)... 2024-03-24T11:16:21.609 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:21 smithi062 ceph-mon[24799]: Reconfiguring daemon prometheus.a on smithi139 2024-03-24T11:16:21.609 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:21 smithi062 ceph-mon[24799]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-03-24T11:16:21.609 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:21 smithi062 ceph-mon[24799]: osdmap e62: 8 total, 8 up, 8 in 2024-03-24T11:16:21.848 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'radosgw-admin zonegroup create --rgw-zonegroup=default --master --default' 2024-03-24T11:16:21.857 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:21 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:21.857 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:21 smithi139 ceph-mon[25719]: Reconfiguring prometheus.a (dependencies changed)... 2024-03-24T11:16:21.857 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:21 smithi139 ceph-mon[25719]: Reconfiguring daemon prometheus.a on smithi139 2024-03-24T11:16:21.857 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:21 smithi139 ceph-mon[25719]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-03-24T11:16:21.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:21 smithi139 ceph-mon[25719]: osdmap e62: 8 total, 8 up, 8 in 2024-03-24T11:16:22.476 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:16:22 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=info ts=2024-03-24T11:16:22.414Z caller=cluster.go:696 component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000557289s 2024-03-24T11:16:23.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:22 smithi062 ceph-mon[24799]: pgmap v52: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:16:23.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:22 smithi062 ceph-mon[24799]: osdmap e63: 8 total, 8 up, 8 in 2024-03-24T11:16:23.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:22 smithi062 ceph-mon[29666]: pgmap v52: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:16:23.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:22 smithi062 ceph-mon[29666]: osdmap e63: 8 total, 8 up, 8 in 2024-03-24T11:16:23.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:22 smithi139 ceph-mon[25719]: pgmap v52: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:16:23.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:22 smithi139 ceph-mon[25719]: osdmap e63: 8 total, 8 up, 8 in 2024-03-24T11:16:23.721 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:16:23 smithi139 systemd[1]: Stopping Ceph prometheus.a for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:16:23.722 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:16:23 smithi139 bash[45985]: Error: no container with name or ID "ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus.a" found: no such container 2024-03-24T11:16:24.006 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:23 smithi139 ceph-mon[25719]: pgmap v54: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:16:24.006 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:16:23 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43865]: ts=2024-03-24T11:16:23.721Z caller=main.go:775 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-03-24T11:16:24.006 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:16:23 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43865]: ts=2024-03-24T11:16:23.721Z caller=main.go:798 level=info msg="Stopping scrape discovery manager..." 2024-03-24T11:16:24.006 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:16:23 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43865]: ts=2024-03-24T11:16:23.721Z caller=main.go:812 level=info msg="Stopping notify discovery manager..." 2024-03-24T11:16:24.006 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:16:23 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43865]: ts=2024-03-24T11:16:23.721Z caller=main.go:834 level=info msg="Stopping scrape manager..." 2024-03-24T11:16:24.006 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:16:23 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43865]: ts=2024-03-24T11:16:23.721Z caller=main.go:794 level=info msg="Scrape discovery manager stopped" 2024-03-24T11:16:24.006 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:16:23 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43865]: ts=2024-03-24T11:16:23.721Z caller=main.go:808 level=info msg="Notify discovery manager stopped" 2024-03-24T11:16:24.007 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:16:23 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43865]: ts=2024-03-24T11:16:23.722Z caller=manager.go:945 level=info component="rule manager" msg="Stopping rule manager..." 2024-03-24T11:16:24.007 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:16:23 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43865]: ts=2024-03-24T11:16:23.722Z caller=main.go:828 level=info msg="Scrape manager stopped" 2024-03-24T11:16:24.007 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:16:23 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43865]: ts=2024-03-24T11:16:23.722Z caller=manager.go:955 level=info component="rule manager" msg="Rule manager stopped" 2024-03-24T11:16:24.007 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:16:23 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43865]: ts=2024-03-24T11:16:23.722Z caller=notifier.go:600 level=info component=notifier msg="Stopping notification manager..." 2024-03-24T11:16:24.007 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:16:23 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43865]: ts=2024-03-24T11:16:23.722Z caller=main.go:1054 level=info msg="Notifier manager stopped" 2024-03-24T11:16:24.007 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:16:23 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[43865]: ts=2024-03-24T11:16:23.722Z caller=main.go:1066 level=info msg="See you next time!" 2024-03-24T11:16:24.007 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:16:23 smithi139 podman[45996]: 2024-03-24 11:16:23.732079002 +0000 UTC m=+0.119433390 container died c1d79fd6d6f1fe40241d869930eabf531f3134eb22a713215b0893fe6f893692 (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-24T11:16:24.140 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:16:24.141 INFO:teuthology.orchestra.run.smithi062.stdout: "id": "01e94997-9c12-49ce-8ce6-d4408cd56555", 2024-03-24T11:16:24.141 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "default", 2024-03-24T11:16:24.141 INFO:teuthology.orchestra.run.smithi062.stdout: "api_name": "default", 2024-03-24T11:16:24.141 INFO:teuthology.orchestra.run.smithi062.stdout: "is_master": "true", 2024-03-24T11:16:24.141 INFO:teuthology.orchestra.run.smithi062.stdout: "endpoints": [], 2024-03-24T11:16:24.141 INFO:teuthology.orchestra.run.smithi062.stdout: "hostnames": [], 2024-03-24T11:16:24.141 INFO:teuthology.orchestra.run.smithi062.stdout: "hostnames_s3website": [], 2024-03-24T11:16:24.141 INFO:teuthology.orchestra.run.smithi062.stdout: "master_zone": "", 2024-03-24T11:16:24.141 INFO:teuthology.orchestra.run.smithi062.stdout: "zones": [], 2024-03-24T11:16:24.141 INFO:teuthology.orchestra.run.smithi062.stdout: "placement_targets": [], 2024-03-24T11:16:24.141 INFO:teuthology.orchestra.run.smithi062.stdout: "default_placement": "", 2024-03-24T11:16:24.141 INFO:teuthology.orchestra.run.smithi062.stdout: "realm_id": "937f0dc2-2b1d-4d64-b23a-bf9f2400aa57", 2024-03-24T11:16:24.141 INFO:teuthology.orchestra.run.smithi062.stdout: "sync_policy": { 2024-03-24T11:16:24.141 INFO:teuthology.orchestra.run.smithi062.stdout: "groups": [] 2024-03-24T11:16:24.142 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-24T11:16:24.142 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:16:24.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:23 smithi062 ceph-mon[24799]: pgmap v54: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:16:24.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:23 smithi062 ceph-mon[29666]: pgmap v54: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:16:24.258 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:16:24 smithi139 podman[45996]: 2024-03-24 11:16:24.213331472 +0000 UTC m=+0.600685913 container cleanup c1d79fd6d6f1fe40241d869930eabf531f3134eb22a713215b0893fe6f893692 (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-24T11:16:24.258 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:16:24 smithi139 bash[45996]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a 2024-03-24T11:16:24.258 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:16:24 smithi139 bash[46023]: Error: no container with name or ID "ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus.a" found: no such container 2024-03-24T11:16:24.800 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:16:24 smithi139 podman[46010]: 2024-03-24 11:16:24.464498517 +0000 UTC m=+0.734678935 container remove c1d79fd6d6f1fe40241d869930eabf531f3134eb22a713215b0893fe6f893692 (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-24T11:16:24.800 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:16:24 smithi139 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@prometheus.a.service: Deactivated successfully. 2024-03-24T11:16:24.800 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:16:24 smithi139 systemd[1]: Stopped Ceph prometheus.a for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:16:24.800 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:16:24 smithi139 systemd[1]: Starting Ceph prometheus.a for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:16:24.800 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:16:24 smithi139 podman[46095]: 2024-03-24 11:16:24.698705634 +0000 UTC m=+0.110710999 container create 3ac3b6a49a6ed3d04961e96124562a615ed65db9e2990c48c93bae547683b1a3 (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-24T11:16:24.800 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:16:24 smithi139 podman[46095]: 2024-03-24 11:16:24.606169833 +0000 UTC m=+0.018175197 image pull 514e6a882f6e74806a5856468489eeff8d7106095557578da96935e4d0ba4d9d quay.io/prometheus/prometheus:v2.33.4 2024-03-24T11:16:24.864 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'radosgw-admin zone create --rgw-zonegroup=default --rgw-zone=z --master --default' 2024-03-24T11:16:25.298 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:16:24 smithi139 podman[46095]: 2024-03-24 11:16:24.909050096 +0000 UTC m=+0.321055459 container init 3ac3b6a49a6ed3d04961e96124562a615ed65db9e2990c48c93bae547683b1a3 (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-24T11:16:25.299 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:16:24 smithi139 podman[46095]: 2024-03-24 11:16:24.911854886 +0000 UTC m=+0.323860256 container start 3ac3b6a49a6ed3d04961e96124562a615ed65db9e2990c48c93bae547683b1a3 (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-24T11:16:25.299 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:16:24 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[46110]: ts=2024-03-24T11:16:24.935Z caller=main.go:475 level=info msg="No time or size retention was set so using the default time retention" duration=15d 2024-03-24T11:16:25.299 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:16:24 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[46110]: ts=2024-03-24T11:16:24.935Z caller=main.go:512 level=info msg="Starting Prometheus" version="(version=2.33.4, branch=HEAD, revision=83032011a5d3e6102624fe58241a374a7201fee8)" 2024-03-24T11:16:25.299 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:16:24 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[46110]: ts=2024-03-24T11:16:24.935Z caller=main.go:517 level=info build_context="(go=go1.17.7, user=root@d13bf69e7be8, date=20220222-16:51:28)" 2024-03-24T11:16:25.299 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:16:24 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[46110]: ts=2024-03-24T11:16:24.935Z caller=main.go:518 level=info host_details="(Linux 5.14.0-430.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024 x86_64 smithi139 (none))" 2024-03-24T11:16:25.299 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:16:24 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[46110]: ts=2024-03-24T11:16:24.935Z caller=main.go:519 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-03-24T11:16:25.299 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:16:24 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[46110]: ts=2024-03-24T11:16:24.935Z caller=main.go:520 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-03-24T11:16:25.299 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:16:24 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[46110]: ts=2024-03-24T11:16:24.937Z caller=web.go:570 level=info component=web msg="Start listening for connections" address=:9095 2024-03-24T11:16:25.299 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:16:24 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[46110]: ts=2024-03-24T11:16:24.937Z caller=main.go:923 level=info msg="Starting TSDB ..." 2024-03-24T11:16:25.299 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:16:24 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[46110]: ts=2024-03-24T11:16:24.938Z caller=tls_config.go:195 level=info component=web msg="TLS is disabled." http2=false 2024-03-24T11:16:25.299 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:16:24 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[46110]: ts=2024-03-24T11:16:24.940Z caller=head.go:493 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-03-24T11:16:25.300 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:16:24 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[46110]: ts=2024-03-24T11:16:24.940Z caller=head.go:527 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=2.086µs 2024-03-24T11:16:25.300 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:16:24 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[46110]: ts=2024-03-24T11:16:24.940Z caller=head.go:533 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-03-24T11:16:25.300 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:16:25 smithi139 bash[46095]: 3ac3b6a49a6ed3d04961e96124562a615ed65db9e2990c48c93bae547683b1a3 2024-03-24T11:16:25.300 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:16:25 smithi139 systemd[1]: Started Ceph prometheus.a for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:16:25.300 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:16:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[46110]: ts=2024-03-24T11:16:25.156Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=3 2024-03-24T11:16:25.300 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:16:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[46110]: ts=2024-03-24T11:16:25.156Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=3 2024-03-24T11:16:25.300 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:16:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[46110]: ts=2024-03-24T11:16:25.298Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=2 maxSegment=3 2024-03-24T11:16:25.300 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:16:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[46110]: ts=2024-03-24T11:16:25.298Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=3 maxSegment=3 2024-03-24T11:16:25.300 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:16:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[46110]: ts=2024-03-24T11:16:25.298Z caller=head.go:610 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=40.228µs wal_replay_duration=357.82627ms total_replay_duration=357.88177ms 2024-03-24T11:16:25.607 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:16:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[46110]: ts=2024-03-24T11:16:25.301Z caller=main.go:944 level=info fs_type=XFS_SUPER_MAGIC 2024-03-24T11:16:25.608 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:16:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[46110]: ts=2024-03-24T11:16:25.301Z caller=main.go:947 level=info msg="TSDB started" 2024-03-24T11:16:25.608 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:16:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[46110]: ts=2024-03-24T11:16:25.301Z caller=main.go:1128 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-03-24T11:16:25.608 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:16:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[46110]: ts=2024-03-24T11:16:25.315Z caller=main.go:1165 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=14.303984ms db_storage=709ns remote_storage=1.139µs web_handler=382ns query_engine=569ns scrape=382.933µs scrape_sd=24.168µs notify=27.182µs notify_sd=7.533µs rules=13.59977ms 2024-03-24T11:16:25.608 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:16:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[46110]: ts=2024-03-24T11:16:25.315Z caller=main.go:896 level=info msg="Server is ready to receive web requests." 2024-03-24T11:16:26.331 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:26 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:26.331 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:26 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-24T11:16:26.331 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:26 smithi062 ceph-mon[24799]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-24T11:16:26.331 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:26 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://172.21.15.62:9093"}]: dispatch 2024-03-24T11:16:26.331 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:26 smithi062 ceph-mon[24799]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://172.21.15.62:9093"}]: dispatch 2024-03-24T11:16:26.331 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:26 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:26.331 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:26 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T11:16:26.331 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:26 smithi062 ceph-mon[24799]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T11:16:26.331 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:26 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:16:26.331 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:26 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:16:26.331 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:26 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:26.331 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:26 smithi062 ceph-mon[24799]: pgmap v55: 33 pgs: 33 active+clean; 577 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s wr, 1 op/s 2024-03-24T11:16:26.332 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:26 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:26.332 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:26 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-24T11:16:26.332 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:26 smithi062 ceph-mon[29666]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-24T11:16:26.332 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:26 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://172.21.15.62:9093"}]: dispatch 2024-03-24T11:16:26.332 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:26 smithi062 ceph-mon[29666]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://172.21.15.62:9093"}]: dispatch 2024-03-24T11:16:26.332 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:26 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:26.332 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:26 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T11:16:26.332 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:26 smithi062 ceph-mon[29666]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T11:16:26.332 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:26 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:16:26.332 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:26 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:16:26.332 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:26 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:26.332 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:26 smithi062 ceph-mon[29666]: pgmap v55: 33 pgs: 33 active+clean; 577 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s wr, 1 op/s 2024-03-24T11:16:26.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:26 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:26.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:26 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-24T11:16:26.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:26 smithi139 ceph-mon[25719]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-24T11:16:26.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:26 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://172.21.15.62:9093"}]: dispatch 2024-03-24T11:16:26.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:26 smithi139 ceph-mon[25719]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://172.21.15.62:9093"}]: dispatch 2024-03-24T11:16:26.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:26 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:26.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:26 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T11:16:26.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:26 smithi139 ceph-mon[25719]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T11:16:26.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:26 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:16:26.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:26 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:16:26.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:26 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:26.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:26 smithi139 ceph-mon[25719]: pgmap v55: 33 pgs: 33 active+clean; 577 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s wr, 1 op/s 2024-03-24T11:16:26.606 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:16:26.606 INFO:teuthology.orchestra.run.smithi062.stdout: "id": "14786e0d-fb4a-4dff-a3b4-79effa223a5f", 2024-03-24T11:16:26.606 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "z", 2024-03-24T11:16:26.606 INFO:teuthology.orchestra.run.smithi062.stdout: "domain_root": "z.rgw.meta:root", 2024-03-24T11:16:26.606 INFO:teuthology.orchestra.run.smithi062.stdout: "control_pool": "z.rgw.control", 2024-03-24T11:16:26.606 INFO:teuthology.orchestra.run.smithi062.stdout: "gc_pool": "z.rgw.log:gc", 2024-03-24T11:16:26.606 INFO:teuthology.orchestra.run.smithi062.stdout: "lc_pool": "z.rgw.log:lc", 2024-03-24T11:16:26.606 INFO:teuthology.orchestra.run.smithi062.stdout: "log_pool": "z.rgw.log", 2024-03-24T11:16:26.606 INFO:teuthology.orchestra.run.smithi062.stdout: "intent_log_pool": "z.rgw.log:intent", 2024-03-24T11:16:26.607 INFO:teuthology.orchestra.run.smithi062.stdout: "usage_log_pool": "z.rgw.log:usage", 2024-03-24T11:16:26.607 INFO:teuthology.orchestra.run.smithi062.stdout: "roles_pool": "z.rgw.meta:roles", 2024-03-24T11:16:26.607 INFO:teuthology.orchestra.run.smithi062.stdout: "reshard_pool": "z.rgw.log:reshard", 2024-03-24T11:16:26.607 INFO:teuthology.orchestra.run.smithi062.stdout: "user_keys_pool": "z.rgw.meta:users.keys", 2024-03-24T11:16:26.607 INFO:teuthology.orchestra.run.smithi062.stdout: "user_email_pool": "z.rgw.meta:users.email", 2024-03-24T11:16:26.607 INFO:teuthology.orchestra.run.smithi062.stdout: "user_swift_pool": "z.rgw.meta:users.swift", 2024-03-24T11:16:26.607 INFO:teuthology.orchestra.run.smithi062.stdout: "user_uid_pool": "z.rgw.meta:users.uid", 2024-03-24T11:16:26.607 INFO:teuthology.orchestra.run.smithi062.stdout: "otp_pool": "z.rgw.otp", 2024-03-24T11:16:26.607 INFO:teuthology.orchestra.run.smithi062.stdout: "system_key": { 2024-03-24T11:16:26.607 INFO:teuthology.orchestra.run.smithi062.stdout: "access_key": "", 2024-03-24T11:16:26.607 INFO:teuthology.orchestra.run.smithi062.stdout: "secret_key": "" 2024-03-24T11:16:26.607 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:16:26.607 INFO:teuthology.orchestra.run.smithi062.stdout: "placement_pools": [ 2024-03-24T11:16:26.608 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-03-24T11:16:26.608 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "default-placement", 2024-03-24T11:16:26.608 INFO:teuthology.orchestra.run.smithi062.stdout: "val": { 2024-03-24T11:16:26.608 INFO:teuthology.orchestra.run.smithi062.stdout: "index_pool": "z.rgw.buckets.index", 2024-03-24T11:16:26.608 INFO:teuthology.orchestra.run.smithi062.stdout: "storage_classes": { 2024-03-24T11:16:26.608 INFO:teuthology.orchestra.run.smithi062.stdout: "STANDARD": { 2024-03-24T11:16:26.608 INFO:teuthology.orchestra.run.smithi062.stdout: "data_pool": "z.rgw.buckets.data" 2024-03-24T11:16:26.608 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-24T11:16:26.608 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:16:26.608 INFO:teuthology.orchestra.run.smithi062.stdout: "data_extra_pool": "z.rgw.buckets.non-ec", 2024-03-24T11:16:26.608 INFO:teuthology.orchestra.run.smithi062.stdout: "index_type": 0 2024-03-24T11:16:26.608 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-24T11:16:26.609 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-24T11:16:26.609 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-03-24T11:16:26.609 INFO:teuthology.orchestra.run.smithi062.stdout: "realm_id": "937f0dc2-2b1d-4d64-b23a-bf9f2400aa57", 2024-03-24T11:16:26.609 INFO:teuthology.orchestra.run.smithi062.stdout: "notif_pool": "z.rgw.log:notif" 2024-03-24T11:16:26.609 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:16:27.836 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'radosgw-admin period update --rgw-realm=r --commit' 2024-03-24T11:16:29.152 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:28 smithi062 ceph-mon[24799]: pgmap v56: 33 pgs: 33 active+clean; 577 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s wr, 1 op/s 2024-03-24T11:16:29.152 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:28 smithi062 ceph-mon[29666]: pgmap v56: 33 pgs: 33 active+clean; 577 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s wr, 1 op/s 2024-03-24T11:16:29.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:28 smithi139 ceph-mon[25719]: pgmap v56: 33 pgs: 33 active+clean; 577 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s wr, 1 op/s 2024-03-24T11:16:30.727 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:16:30 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=info ts=2024-03-24T11:16:30.417Z caller=cluster.go:688 component=cluster msg="gossip settled; proceeding" elapsed=10.003139112s 2024-03-24T11:16:31.089 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:30 smithi062 ceph-mon[24799]: pgmap v57: 33 pgs: 33 active+clean; 579 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 803 B/s rd, 1.8 KiB/s wr, 3 op/s 2024-03-24T11:16:31.090 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:30 smithi062 ceph-mon[24799]: osdmap e64: 8 total, 8 up, 8 in 2024-03-24T11:16:31.090 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:30 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/192276467' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.log","app": "rgw"}]: dispatch 2024-03-24T11:16:31.090 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:30 smithi062 ceph-mon[29666]: pgmap v57: 33 pgs: 33 active+clean; 579 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 803 B/s rd, 1.8 KiB/s wr, 3 op/s 2024-03-24T11:16:31.090 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:30 smithi062 ceph-mon[29666]: osdmap e64: 8 total, 8 up, 8 in 2024-03-24T11:16:31.090 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:30 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/192276467' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.log","app": "rgw"}]: dispatch 2024-03-24T11:16:31.107 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:16:30 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:16:30] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-03-24T11:16:31.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:30 smithi139 ceph-mon[25719]: pgmap v57: 33 pgs: 33 active+clean; 579 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 803 B/s rd, 1.8 KiB/s wr, 3 op/s 2024-03-24T11:16:31.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:30 smithi139 ceph-mon[25719]: osdmap e64: 8 total, 8 up, 8 in 2024-03-24T11:16:31.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:30 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/192276467' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.log","app": "rgw"}]: dispatch 2024-03-24T11:16:32.226 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:31 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/192276467' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.log","app": "rgw"}]': finished 2024-03-24T11:16:32.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:31 smithi062 ceph-mon[29666]: osdmap e65: 8 total, 8 up, 8 in 2024-03-24T11:16:32.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:31 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/192276467' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.log","app": "rgw"}]': finished 2024-03-24T11:16:32.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:31 smithi062 ceph-mon[24799]: osdmap e65: 8 total, 8 up, 8 in 2024-03-24T11:16:32.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:31 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/192276467' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.log","app": "rgw"}]': finished 2024-03-24T11:16:32.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:31 smithi139 ceph-mon[25719]: osdmap e65: 8 total, 8 up, 8 in 2024-03-24T11:16:33.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:32 smithi062 ceph-mon[24799]: pgmap v60: 65 pgs: 32 unknown, 33 active+clean; 579 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 2.0 KiB/s wr, 3 op/s 2024-03-24T11:16:33.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:32 smithi062 ceph-mon[24799]: osdmap e66: 8 total, 8 up, 8 in 2024-03-24T11:16:33.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:32 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/192276467' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.control","app": "rgw"}]: dispatch 2024-03-24T11:16:33.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:32 smithi062 ceph-mon[29666]: pgmap v60: 65 pgs: 32 unknown, 33 active+clean; 579 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 2.0 KiB/s wr, 3 op/s 2024-03-24T11:16:33.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:32 smithi062 ceph-mon[29666]: osdmap e66: 8 total, 8 up, 8 in 2024-03-24T11:16:33.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:32 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/192276467' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.control","app": "rgw"}]: dispatch 2024-03-24T11:16:33.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:32 smithi139 ceph-mon[25719]: pgmap v60: 65 pgs: 32 unknown, 33 active+clean; 579 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 2.0 KiB/s wr, 3 op/s 2024-03-24T11:16:33.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:32 smithi139 ceph-mon[25719]: osdmap e66: 8 total, 8 up, 8 in 2024-03-24T11:16:33.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:32 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/192276467' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.control","app": "rgw"}]: dispatch 2024-03-24T11:16:33.727 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:16:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.62 - - [24/Mar/2024:11:16:33] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:16:33.727 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:16:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.62 - - [24/Mar/2024:11:16:33] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:16:34.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:33 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/192276467' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.control","app": "rgw"}]': finished 2024-03-24T11:16:34.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:33 smithi062 ceph-mon[24799]: osdmap e67: 8 total, 8 up, 8 in 2024-03-24T11:16:34.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:33 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:34.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:33 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:34.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:33 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:34.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:33 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:34.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:33 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/192276467' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.control","app": "rgw"}]': finished 2024-03-24T11:16:34.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:33 smithi062 ceph-mon[29666]: osdmap e67: 8 total, 8 up, 8 in 2024-03-24T11:16:34.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:33 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:34.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:33 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:34.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:33 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:34.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:33 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:34.259 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:33 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/192276467' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.control","app": "rgw"}]': finished 2024-03-24T11:16:34.259 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:33 smithi139 ceph-mon[25719]: osdmap e67: 8 total, 8 up, 8 in 2024-03-24T11:16:34.259 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:33 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:34.259 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:33 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:34.259 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:33 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:34.259 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:33 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:35.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:34 smithi062 ceph-mon[29666]: pgmap v63: 97 pgs: 64 unknown, 33 active+clean; 579 KiB data, 49 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:16:35.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:34 smithi062 ceph-mon[29666]: osdmap e68: 8 total, 8 up, 8 in 2024-03-24T11:16:35.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:34 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/3546635726' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]: dispatch 2024-03-24T11:16:35.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:34 smithi062 ceph-mon[29666]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]: dispatch 2024-03-24T11:16:35.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:34 smithi062 ceph-mon[24799]: pgmap v63: 97 pgs: 64 unknown, 33 active+clean; 579 KiB data, 49 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:16:35.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:34 smithi062 ceph-mon[24799]: osdmap e68: 8 total, 8 up, 8 in 2024-03-24T11:16:35.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:34 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/3546635726' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]: dispatch 2024-03-24T11:16:35.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:34 smithi062 ceph-mon[24799]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]: dispatch 2024-03-24T11:16:35.241 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:34 smithi139 ceph-mon[25719]: pgmap v63: 97 pgs: 64 unknown, 33 active+clean; 579 KiB data, 49 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:16:35.241 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:34 smithi139 ceph-mon[25719]: osdmap e68: 8 total, 8 up, 8 in 2024-03-24T11:16:35.241 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:34 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/3546635726' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]: dispatch 2024-03-24T11:16:35.241 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:34 smithi139 ceph-mon[25719]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]: dispatch 2024-03-24T11:16:36.196 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:35 smithi139 ceph-mon[25719]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]': finished 2024-03-24T11:16:36.196 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:35 smithi139 ceph-mon[25719]: osdmap e69: 8 total, 8 up, 8 in 2024-03-24T11:16:36.196 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:35 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/3546635726' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-03-24T11:16:36.196 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:35 smithi139 ceph-mon[25719]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-03-24T11:16:36.196 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:35 smithi139 ceph-mon[25719]: pgmap v66: 129 pgs: 32 unknown, 97 active+clean; 579 KiB data, 51 MiB used, 715 GiB / 715 GiB avail; 6.0 KiB/s rd, 511 B/s wr, 7 op/s 2024-03-24T11:16:36.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:35 smithi062 ceph-mon[24799]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]': finished 2024-03-24T11:16:36.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:35 smithi062 ceph-mon[24799]: osdmap e69: 8 total, 8 up, 8 in 2024-03-24T11:16:36.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:35 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/3546635726' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-03-24T11:16:36.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:35 smithi062 ceph-mon[24799]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-03-24T11:16:36.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:35 smithi062 ceph-mon[24799]: pgmap v66: 129 pgs: 32 unknown, 97 active+clean; 579 KiB data, 51 MiB used, 715 GiB / 715 GiB avail; 6.0 KiB/s rd, 511 B/s wr, 7 op/s 2024-03-24T11:16:36.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:35 smithi062 ceph-mon[29666]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]': finished 2024-03-24T11:16:36.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:35 smithi062 ceph-mon[29666]: osdmap e69: 8 total, 8 up, 8 in 2024-03-24T11:16:36.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:35 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/3546635726' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-03-24T11:16:36.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:35 smithi062 ceph-mon[29666]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-03-24T11:16:36.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:35 smithi062 ceph-mon[29666]: pgmap v66: 129 pgs: 32 unknown, 97 active+clean; 579 KiB data, 51 MiB used, 715 GiB / 715 GiB avail; 6.0 KiB/s rd, 511 B/s wr, 7 op/s 2024-03-24T11:16:37.005 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:16:37.005 INFO:teuthology.orchestra.run.smithi062.stdout: "id": "6730a3c9-615d-4b3d-a519-4a8f82eabf03", 2024-03-24T11:16:37.005 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 1, 2024-03-24T11:16:37.005 INFO:teuthology.orchestra.run.smithi062.stdout: "predecessor_uuid": "f1091927-c182-4e18-b336-f2dc08906f60", 2024-03-24T11:16:37.005 INFO:teuthology.orchestra.run.smithi062.stdout: "sync_status": [], 2024-03-24T11:16:37.006 INFO:teuthology.orchestra.run.smithi062.stdout: "period_map": { 2024-03-24T11:16:37.006 INFO:teuthology.orchestra.run.smithi062.stdout: "id": "6730a3c9-615d-4b3d-a519-4a8f82eabf03", 2024-03-24T11:16:37.006 INFO:teuthology.orchestra.run.smithi062.stdout: "zonegroups": [ 2024-03-24T11:16:37.006 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-03-24T11:16:37.006 INFO:teuthology.orchestra.run.smithi062.stdout: "id": "01e94997-9c12-49ce-8ce6-d4408cd56555", 2024-03-24T11:16:37.006 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "default", 2024-03-24T11:16:37.006 INFO:teuthology.orchestra.run.smithi062.stdout: "api_name": "default", 2024-03-24T11:16:37.006 INFO:teuthology.orchestra.run.smithi062.stdout: "is_master": "true", 2024-03-24T11:16:37.006 INFO:teuthology.orchestra.run.smithi062.stdout: "endpoints": [], 2024-03-24T11:16:37.006 INFO:teuthology.orchestra.run.smithi062.stdout: "hostnames": [], 2024-03-24T11:16:37.006 INFO:teuthology.orchestra.run.smithi062.stdout: "hostnames_s3website": [], 2024-03-24T11:16:37.006 INFO:teuthology.orchestra.run.smithi062.stdout: "master_zone": "14786e0d-fb4a-4dff-a3b4-79effa223a5f", 2024-03-24T11:16:37.006 INFO:teuthology.orchestra.run.smithi062.stdout: "zones": [ 2024-03-24T11:16:37.006 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-03-24T11:16:37.007 INFO:teuthology.orchestra.run.smithi062.stdout: "id": "14786e0d-fb4a-4dff-a3b4-79effa223a5f", 2024-03-24T11:16:37.007 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "z", 2024-03-24T11:16:37.007 INFO:teuthology.orchestra.run.smithi062.stdout: "endpoints": [], 2024-03-24T11:16:37.007 INFO:teuthology.orchestra.run.smithi062.stdout: "log_meta": "false", 2024-03-24T11:16:37.007 INFO:teuthology.orchestra.run.smithi062.stdout: "log_data": "false", 2024-03-24T11:16:37.007 INFO:teuthology.orchestra.run.smithi062.stdout: "bucket_index_max_shards": 11, 2024-03-24T11:16:37.007 INFO:teuthology.orchestra.run.smithi062.stdout: "read_only": "false", 2024-03-24T11:16:37.007 INFO:teuthology.orchestra.run.smithi062.stdout: "tier_type": "", 2024-03-24T11:16:37.007 INFO:teuthology.orchestra.run.smithi062.stdout: "sync_from_all": "true", 2024-03-24T11:16:37.007 INFO:teuthology.orchestra.run.smithi062.stdout: "sync_from": [], 2024-03-24T11:16:37.007 INFO:teuthology.orchestra.run.smithi062.stdout: "redirect_zone": "" 2024-03-24T11:16:37.007 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-24T11:16:37.007 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-03-24T11:16:37.007 INFO:teuthology.orchestra.run.smithi062.stdout: "placement_targets": [ 2024-03-24T11:16:37.007 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-03-24T11:16:37.008 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "default-placement", 2024-03-24T11:16:37.008 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-03-24T11:16:37.008 INFO:teuthology.orchestra.run.smithi062.stdout: "storage_classes": [ 2024-03-24T11:16:37.008 INFO:teuthology.orchestra.run.smithi062.stdout: "STANDARD" 2024-03-24T11:16:37.008 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-03-24T11:16:37.008 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-24T11:16:37.008 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-03-24T11:16:37.008 INFO:teuthology.orchestra.run.smithi062.stdout: "default_placement": "default-placement", 2024-03-24T11:16:37.008 INFO:teuthology.orchestra.run.smithi062.stdout: "realm_id": "937f0dc2-2b1d-4d64-b23a-bf9f2400aa57", 2024-03-24T11:16:37.008 INFO:teuthology.orchestra.run.smithi062.stdout: "sync_policy": { 2024-03-24T11:16:37.008 INFO:teuthology.orchestra.run.smithi062.stdout: "groups": [] 2024-03-24T11:16:37.008 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-24T11:16:37.008 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-24T11:16:37.008 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-03-24T11:16:37.008 INFO:teuthology.orchestra.run.smithi062.stdout: "short_zone_ids": [ 2024-03-24T11:16:37.009 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-03-24T11:16:37.009 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "14786e0d-fb4a-4dff-a3b4-79effa223a5f", 2024-03-24T11:16:37.009 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 551977852 2024-03-24T11:16:37.009 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-24T11:16:37.009 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-03-24T11:16:37.009 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:16:37.009 INFO:teuthology.orchestra.run.smithi062.stdout: "master_zonegroup": "01e94997-9c12-49ce-8ce6-d4408cd56555", 2024-03-24T11:16:37.009 INFO:teuthology.orchestra.run.smithi062.stdout: "master_zone": "14786e0d-fb4a-4dff-a3b4-79effa223a5f", 2024-03-24T11:16:37.009 INFO:teuthology.orchestra.run.smithi062.stdout: "period_config": { 2024-03-24T11:16:37.009 INFO:teuthology.orchestra.run.smithi062.stdout: "bucket_quota": { 2024-03-24T11:16:37.009 INFO:teuthology.orchestra.run.smithi062.stdout: "enabled": false, 2024-03-24T11:16:37.009 INFO:teuthology.orchestra.run.smithi062.stdout: "check_on_raw": false, 2024-03-24T11:16:37.009 INFO:teuthology.orchestra.run.smithi062.stdout: "max_size": -1, 2024-03-24T11:16:37.009 INFO:teuthology.orchestra.run.smithi062.stdout: "max_size_kb": 0, 2024-03-24T11:16:37.009 INFO:teuthology.orchestra.run.smithi062.stdout: "max_objects": -1 2024-03-24T11:16:37.010 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:16:37.010 INFO:teuthology.orchestra.run.smithi062.stdout: "user_quota": { 2024-03-24T11:16:37.010 INFO:teuthology.orchestra.run.smithi062.stdout: "enabled": false, 2024-03-24T11:16:37.010 INFO:teuthology.orchestra.run.smithi062.stdout: "check_on_raw": false, 2024-03-24T11:16:37.010 INFO:teuthology.orchestra.run.smithi062.stdout: "max_size": -1, 2024-03-24T11:16:37.010 INFO:teuthology.orchestra.run.smithi062.stdout: "max_size_kb": 0, 2024-03-24T11:16:37.010 INFO:teuthology.orchestra.run.smithi062.stdout: "max_objects": -1 2024-03-24T11:16:37.010 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:16:37.010 INFO:teuthology.orchestra.run.smithi062.stdout: "user_ratelimit": { 2024-03-24T11:16:37.010 INFO:teuthology.orchestra.run.smithi062.stdout: "max_read_ops": 0, 2024-03-24T11:16:37.010 INFO:teuthology.orchestra.run.smithi062.stdout: "max_write_ops": 0, 2024-03-24T11:16:37.010 INFO:teuthology.orchestra.run.smithi062.stdout: "max_read_bytes": 0, 2024-03-24T11:16:37.010 INFO:teuthology.orchestra.run.smithi062.stdout: "max_write_bytes": 0, 2024-03-24T11:16:37.010 INFO:teuthology.orchestra.run.smithi062.stdout: "enabled": false 2024-03-24T11:16:37.010 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:16:37.011 INFO:teuthology.orchestra.run.smithi062.stdout: "bucket_ratelimit": { 2024-03-24T11:16:37.011 INFO:teuthology.orchestra.run.smithi062.stdout: "max_read_ops": 0, 2024-03-24T11:16:37.011 INFO:teuthology.orchestra.run.smithi062.stdout: "max_write_ops": 0, 2024-03-24T11:16:37.011 INFO:teuthology.orchestra.run.smithi062.stdout: "max_read_bytes": 0, 2024-03-24T11:16:37.011 INFO:teuthology.orchestra.run.smithi062.stdout: "max_write_bytes": 0, 2024-03-24T11:16:37.011 INFO:teuthology.orchestra.run.smithi062.stdout: "enabled": false 2024-03-24T11:16:37.011 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:16:37.011 INFO:teuthology.orchestra.run.smithi062.stdout: "anonymous_ratelimit": { 2024-03-24T11:16:37.011 INFO:teuthology.orchestra.run.smithi062.stdout: "max_read_ops": 0, 2024-03-24T11:16:37.011 INFO:teuthology.orchestra.run.smithi062.stdout: "max_write_ops": 0, 2024-03-24T11:16:37.011 INFO:teuthology.orchestra.run.smithi062.stdout: "max_read_bytes": 0, 2024-03-24T11:16:37.011 INFO:teuthology.orchestra.run.smithi062.stdout: "max_write_bytes": 0, 2024-03-24T11:16:37.011 INFO:teuthology.orchestra.run.smithi062.stdout: "enabled": false 2024-03-24T11:16:37.011 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-24T11:16:37.011 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:16:37.012 INFO:teuthology.orchestra.run.smithi062.stdout: "realm_id": "937f0dc2-2b1d-4d64-b23a-bf9f2400aa57", 2024-03-24T11:16:37.012 INFO:teuthology.orchestra.run.smithi062.stdout: "realm_name": "r", 2024-03-24T11:16:37.012 INFO:teuthology.orchestra.run.smithi062.stdout: "realm_epoch": 2 2024-03-24T11:16:37.012 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:16:37.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:36 smithi062 ceph-mon[29666]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-03-24T11:16:37.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:36 smithi062 ceph-mon[29666]: osdmap e70: 8 total, 8 up, 8 in 2024-03-24T11:16:37.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:36 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/3546635726' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_num_min", "val": "8"}]: dispatch 2024-03-24T11:16:37.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:36 smithi062 ceph-mon[29666]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_num_min", "val": "8"}]: dispatch 2024-03-24T11:16:37.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:36 smithi062 ceph-mon[24799]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-03-24T11:16:37.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:36 smithi062 ceph-mon[24799]: osdmap e70: 8 total, 8 up, 8 in 2024-03-24T11:16:37.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:36 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/3546635726' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_num_min", "val": "8"}]: dispatch 2024-03-24T11:16:37.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:36 smithi062 ceph-mon[24799]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_num_min", "val": "8"}]: dispatch 2024-03-24T11:16:37.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:36 smithi139 ceph-mon[25719]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-03-24T11:16:37.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:36 smithi139 ceph-mon[25719]: osdmap e70: 8 total, 8 up, 8 in 2024-03-24T11:16:37.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:36 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/3546635726' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_num_min", "val": "8"}]: dispatch 2024-03-24T11:16:37.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:36 smithi139 ceph-mon[25719]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_num_min", "val": "8"}]: dispatch 2024-03-24T11:16:37.709 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph orch apply rgw foo --realm r --zone z --placement=2 --port=8000' 2024-03-24T11:16:38.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:37 smithi139 ceph-mon[25719]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_num_min", "val": "8"}]': finished 2024-03-24T11:16:38.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:37 smithi139 ceph-mon[25719]: osdmap e71: 8 total, 8 up, 8 in 2024-03-24T11:16:38.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:37 smithi139 ceph-mon[25719]: pgmap v69: 129 pgs: 32 unknown, 97 active+clean; 579 KiB data, 51 MiB used, 715 GiB / 715 GiB avail; 6.0 KiB/s rd, 511 B/s wr, 7 op/s 2024-03-24T11:16:38.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:37 smithi062 ceph-mon[24799]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_num_min", "val": "8"}]': finished 2024-03-24T11:16:38.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:37 smithi062 ceph-mon[24799]: osdmap e71: 8 total, 8 up, 8 in 2024-03-24T11:16:38.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:37 smithi062 ceph-mon[24799]: pgmap v69: 129 pgs: 32 unknown, 97 active+clean; 579 KiB data, 51 MiB used, 715 GiB / 715 GiB avail; 6.0 KiB/s rd, 511 B/s wr, 7 op/s 2024-03-24T11:16:38.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:37 smithi062 ceph-mon[29666]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_num_min", "val": "8"}]': finished 2024-03-24T11:16:38.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:37 smithi062 ceph-mon[29666]: osdmap e71: 8 total, 8 up, 8 in 2024-03-24T11:16:38.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:37 smithi062 ceph-mon[29666]: pgmap v69: 129 pgs: 32 unknown, 97 active+clean; 579 KiB data, 51 MiB used, 715 GiB / 715 GiB avail; 6.0 KiB/s rd, 511 B/s wr, 7 op/s 2024-03-24T11:16:39.187 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:16:38 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:16:38] "GET /metrics HTTP/1.1" 200 195082 "" "Prometheus/2.33.4" 2024-03-24T11:16:39.606 INFO:teuthology.orchestra.run.smithi062.stdout:Scheduled rgw.foo update... 2024-03-24T11:16:39.927 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:16:39 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.139 - - [24/Mar/2024:11:16:39] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:16:40.343 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph orch apply rgw smpl' 2024-03-24T11:16:40.739 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:40 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:40.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:40 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:40.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:40 smithi062 ceph-mon[24799]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo", "realm": "r", "zone": "z", "placement": "2", "port": 8000, "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:16:40.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:40 smithi062 ceph-mon[24799]: Saving service rgw.foo spec with placement count:2 2024-03-24T11:16:40.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:40 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:40.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:40 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:16:40.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:40 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:16:40.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:40 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:40.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:40 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:40.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:40 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:40.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:40 smithi062 ceph-mon[24799]: Saving service rgw.foo spec with placement count:2 2024-03-24T11:16:40.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:40 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:40.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:40 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi062.sacpds", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-03-24T11:16:40.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:40 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi062.sacpds", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-03-24T11:16:40.741 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:40 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:40.741 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:40 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:16:40.741 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:40 smithi062 ceph-mon[24799]: Deploying daemon rgw.foo.smithi062.sacpds on smithi062 2024-03-24T11:16:40.741 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:40 smithi062 ceph-mon[24799]: pgmap v70: 129 pgs: 129 active+clean; 581 KiB data, 52 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-03-24T11:16:40.741 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:40 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:40.741 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:40 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:40.741 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:40 smithi062 ceph-mon[29666]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo", "realm": "r", "zone": "z", "placement": "2", "port": 8000, "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:16:40.741 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:40 smithi062 ceph-mon[29666]: Saving service rgw.foo spec with placement count:2 2024-03-24T11:16:40.741 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:40 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:40.741 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:40 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:16:40.741 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:40 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:16:40.742 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:40 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:40.742 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:40 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:40.742 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:40 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:40.742 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:40 smithi062 ceph-mon[29666]: Saving service rgw.foo spec with placement count:2 2024-03-24T11:16:40.742 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:40 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:40.742 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:40 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi062.sacpds", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-03-24T11:16:40.742 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:40 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi062.sacpds", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-03-24T11:16:40.742 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:40 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:40.742 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:40 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:16:40.742 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:40 smithi062 ceph-mon[29666]: Deploying daemon rgw.foo.smithi062.sacpds on smithi062 2024-03-24T11:16:40.742 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:40 smithi062 ceph-mon[29666]: pgmap v70: 129 pgs: 129 active+clean; 581 KiB data, 52 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-03-24T11:16:40.777 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:40 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:40.777 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:40 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:40.778 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:40 smithi139 ceph-mon[25719]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo", "realm": "r", "zone": "z", "placement": "2", "port": 8000, "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:16:40.778 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:40 smithi139 ceph-mon[25719]: Saving service rgw.foo spec with placement count:2 2024-03-24T11:16:40.778 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:40 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:40.778 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:40 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:16:40.778 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:40 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:16:40.778 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:40 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:40.778 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:40 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:40.778 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:40 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:40.778 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:40 smithi139 ceph-mon[25719]: Saving service rgw.foo spec with placement count:2 2024-03-24T11:16:40.778 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:40 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:40.778 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:40 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi062.sacpds", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-03-24T11:16:40.778 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:40 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi062.sacpds", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-03-24T11:16:40.778 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:40 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:40.779 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:40 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:16:40.779 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:40 smithi139 ceph-mon[25719]: Deploying daemon rgw.foo.smithi062.sacpds on smithi062 2024-03-24T11:16:40.779 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:40 smithi139 ceph-mon[25719]: pgmap v70: 129 pgs: 129 active+clean; 581 KiB data, 52 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-03-24T11:16:41.108 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:16:40 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:16:40] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-03-24T11:16:43.208 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:42 smithi062 ceph-mon[24799]: pgmap v71: 129 pgs: 129 active+clean; 581 KiB data, 52 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.0 KiB/s wr, 2 op/s 2024-03-24T11:16:43.208 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:42 smithi062 ceph-mon[29666]: pgmap v71: 129 pgs: 129 active+clean; 581 KiB data, 52 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.0 KiB/s wr, 2 op/s 2024-03-24T11:16:43.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:42 smithi139 ceph-mon[25719]: pgmap v71: 129 pgs: 129 active+clean; 581 KiB data, 52 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.0 KiB/s wr, 2 op/s 2024-03-24T11:16:43.728 INFO:teuthology.orchestra.run.smithi062.stdout:Scheduled rgw.smpl update... 2024-03-24T11:16:43.870 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:16:43 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:16:43.521Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.139:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.139 because it doesn't contain any IP SANs" 2024-03-24T11:16:43.870 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:16:43 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:16:43.521Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.62:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.62 because it doesn't contain any IP SANs" 2024-03-24T11:16:44.587 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph osd pool create foo' 2024-03-24T11:16:44.851 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:44 smithi062 ceph-mon[24799]: from='client.24595 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "smpl", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:16:44.851 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:44 smithi062 ceph-mon[24799]: Saving service rgw.smpl spec with placement count:2 2024-03-24T11:16:44.851 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:44 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:44.851 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:44 smithi062 ceph-mon[24799]: pgmap v72: 129 pgs: 129 active+clean; 581 KiB data, 52 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 895 B/s wr, 2 op/s 2024-03-24T11:16:44.851 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:44 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:44.851 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:44 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi139.yljesh", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-03-24T11:16:44.852 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:44 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi139.yljesh", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-03-24T11:16:44.852 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:44 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:44.852 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:44 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:16:44.852 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:44 smithi062 ceph-mon[29666]: from='client.24595 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "smpl", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:16:44.852 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:44 smithi062 ceph-mon[29666]: Saving service rgw.smpl spec with placement count:2 2024-03-24T11:16:44.852 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:44 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:44.852 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:44 smithi062 ceph-mon[29666]: pgmap v72: 129 pgs: 129 active+clean; 581 KiB data, 52 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 895 B/s wr, 2 op/s 2024-03-24T11:16:44.852 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:44 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:44.852 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:44 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi139.yljesh", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-03-24T11:16:44.852 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:44 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi139.yljesh", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-03-24T11:16:44.852 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:44 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:44.852 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:44 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:16:45.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:44 smithi139 ceph-mon[25719]: from='client.24595 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "smpl", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:16:45.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:44 smithi139 ceph-mon[25719]: Saving service rgw.smpl spec with placement count:2 2024-03-24T11:16:45.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:44 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:45.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:44 smithi139 ceph-mon[25719]: pgmap v72: 129 pgs: 129 active+clean; 581 KiB data, 52 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 895 B/s wr, 2 op/s 2024-03-24T11:16:45.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:44 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:45.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:44 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi139.yljesh", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-03-24T11:16:45.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:44 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi139.yljesh", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-03-24T11:16:45.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:44 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:45.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:44 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:16:45.976 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:45 smithi062 ceph-mon[24799]: Deploying daemon rgw.foo.smithi139.yljesh on smithi139 2024-03-24T11:16:45.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:45 smithi062 ceph-mon[29666]: Deploying daemon rgw.foo.smithi139.yljesh on smithi139 2024-03-24T11:16:46.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:45 smithi139 ceph-mon[25719]: Deploying daemon rgw.foo.smithi139.yljesh on smithi139 2024-03-24T11:16:46.986 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'foo' created 2024-03-24T11:16:47.173 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:46 smithi139 ceph-mon[25719]: pgmap v73: 129 pgs: 129 active+clean; 583 KiB data, 52 MiB used, 715 GiB / 715 GiB avail; 2.3 KiB/s rd, 2.9 KiB/s wr, 12 op/s 2024-03-24T11:16:47.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:46 smithi062 ceph-mon[24799]: pgmap v73: 129 pgs: 129 active+clean; 583 KiB data, 52 MiB used, 715 GiB / 715 GiB avail; 2.3 KiB/s rd, 2.9 KiB/s wr, 12 op/s 2024-03-24T11:16:47.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:16:46 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.139 - - [24/Mar/2024:11:16:46] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:16:47.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:46 smithi062 ceph-mon[29666]: pgmap v73: 129 pgs: 129 active+clean; 583 KiB data, 52 MiB used, 715 GiB / 715 GiB avail; 2.3 KiB/s rd, 2.9 KiB/s wr, 12 op/s 2024-03-24T11:16:47.731 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'rbd pool init foo' 2024-03-24T11:16:47.960 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:47 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/2153247821' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "foo"}]: dispatch 2024-03-24T11:16:47.960 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:47 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:47.960 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:47 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:47.960 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:47 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/2153247821' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "foo"}]': finished 2024-03-24T11:16:47.960 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:47 smithi062 ceph-mon[24799]: osdmap e72: 8 total, 8 up, 8 in 2024-03-24T11:16:47.960 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:47 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-24T11:16:47.961 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:47 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/2153247821' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "foo"}]: dispatch 2024-03-24T11:16:47.961 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:47 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:47.961 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:47 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:47.961 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:47 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/2153247821' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "foo"}]': finished 2024-03-24T11:16:47.961 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:47 smithi062 ceph-mon[29666]: osdmap e72: 8 total, 8 up, 8 in 2024-03-24T11:16:47.961 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:47 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-24T11:16:48.194 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:47 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/2153247821' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "foo"}]: dispatch 2024-03-24T11:16:48.194 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:47 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:48.194 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:47 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:48.194 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:47 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/2153247821' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "foo"}]': finished 2024-03-24T11:16:48.194 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:47 smithi139 ceph-mon[25719]: osdmap e72: 8 total, 8 up, 8 in 2024-03-24T11:16:48.194 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:47 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-24T11:16:48.977 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:16:48 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:16:48] "GET /metrics HTTP/1.1" 200 203616 "" "Prometheus/2.33.4" 2024-03-24T11:16:48.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:48 smithi062 ceph-mon[24799]: pgmap v75: 161 pgs: 32 unknown, 129 active+clean; 583 KiB data, 52 MiB used, 715 GiB / 715 GiB avail; 2.3 KiB/s rd, 2.9 KiB/s wr, 12 op/s 2024-03-24T11:16:48.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:48 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-24T11:16:48.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:48 smithi062 ceph-mon[24799]: osdmap e73: 8 total, 8 up, 8 in 2024-03-24T11:16:48.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:48 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:48.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:48 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:16:48.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:48 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:16:48.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:48 smithi062 ceph-mon[29666]: pgmap v75: 161 pgs: 32 unknown, 129 active+clean; 583 KiB data, 52 MiB used, 715 GiB / 715 GiB avail; 2.3 KiB/s rd, 2.9 KiB/s wr, 12 op/s 2024-03-24T11:16:48.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:48 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-24T11:16:48.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:48 smithi062 ceph-mon[29666]: osdmap e73: 8 total, 8 up, 8 in 2024-03-24T11:16:48.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:48 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:48.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:48 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:16:48.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:48 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:16:49.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:48 smithi139 ceph-mon[25719]: pgmap v75: 161 pgs: 32 unknown, 129 active+clean; 583 KiB data, 52 MiB used, 715 GiB / 715 GiB avail; 2.3 KiB/s rd, 2.9 KiB/s wr, 12 op/s 2024-03-24T11:16:49.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:48 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-24T11:16:49.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:48 smithi139 ceph-mon[25719]: osdmap e73: 8 total, 8 up, 8 in 2024-03-24T11:16:49.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:48 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:49.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:48 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:16:49.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:48 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:16:50.071 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:49 smithi062 ceph-mon[24799]: Checking dashboard <-> RGW credentials 2024-03-24T11:16:50.071 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:49 smithi062 ceph-mon[24799]: osdmap e74: 8 total, 8 up, 8 in 2024-03-24T11:16:50.072 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:49 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:50.072 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:49 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:50.072 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:49 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:50.072 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:49 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:16:50.072 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:49 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:16:50.072 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:49 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:50.072 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:49 smithi062 ceph-mon[29666]: Checking dashboard <-> RGW credentials 2024-03-24T11:16:50.072 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:49 smithi062 ceph-mon[29666]: osdmap e74: 8 total, 8 up, 8 in 2024-03-24T11:16:50.072 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:49 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:50.072 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:49 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:50.072 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:49 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:50.072 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:49 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:16:50.073 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:49 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:16:50.073 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:49 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:50.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:49 smithi139 ceph-mon[25719]: Checking dashboard <-> RGW credentials 2024-03-24T11:16:50.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:49 smithi139 ceph-mon[25719]: osdmap e74: 8 total, 8 up, 8 in 2024-03-24T11:16:50.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:49 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:50.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:49 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:50.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:49 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:50.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:49 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:16:50.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:49 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:16:50.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:49 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:51.053 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:50 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:16:51.053 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:50 smithi062 ceph-mon[24799]: Checking dashboard <-> RGW credentials 2024-03-24T11:16:51.053 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:50 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:16:51.053 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:50 smithi062 ceph-mon[24799]: pgmap v78: 161 pgs: 161 active+clean; 584 KiB data, 54 MiB used, 715 GiB / 715 GiB avail; 95 KiB/s rd, 5.7 KiB/s wr, 213 op/s 2024-03-24T11:16:51.053 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:50 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/3260560837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "foo","app": "rbd"}]: dispatch 2024-03-24T11:16:51.054 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:50 smithi062 ceph-mon[24799]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "foo","app": "rbd"}]: dispatch 2024-03-24T11:16:51.054 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:50 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:16:51.054 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:50 smithi062 ceph-mon[29666]: Checking dashboard <-> RGW credentials 2024-03-24T11:16:51.054 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:50 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:16:51.054 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:50 smithi062 ceph-mon[29666]: pgmap v78: 161 pgs: 161 active+clean; 584 KiB data, 54 MiB used, 715 GiB / 715 GiB avail; 95 KiB/s rd, 5.7 KiB/s wr, 213 op/s 2024-03-24T11:16:51.054 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:50 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/3260560837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "foo","app": "rbd"}]: dispatch 2024-03-24T11:16:51.054 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:50 smithi062 ceph-mon[29666]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "foo","app": "rbd"}]: dispatch 2024-03-24T11:16:51.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:50 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:16:51.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:50 smithi139 ceph-mon[25719]: Checking dashboard <-> RGW credentials 2024-03-24T11:16:51.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:50 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:16:51.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:50 smithi139 ceph-mon[25719]: pgmap v78: 161 pgs: 161 active+clean; 584 KiB data, 54 MiB used, 715 GiB / 715 GiB avail; 95 KiB/s rd, 5.7 KiB/s wr, 213 op/s 2024-03-24T11:16:51.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:50 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/3260560837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "foo","app": "rbd"}]: dispatch 2024-03-24T11:16:51.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:50 smithi139 ceph-mon[25719]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "foo","app": "rbd"}]: dispatch 2024-03-24T11:16:51.110 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:16:50 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:16:50] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-03-24T11:16:52.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:51 smithi139 ceph-mon[25719]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "foo","app": "rbd"}]': finished 2024-03-24T11:16:52.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:51 smithi139 ceph-mon[25719]: osdmap e75: 8 total, 8 up, 8 in 2024-03-24T11:16:52.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:51 smithi139 ceph-mon[25719]: pgmap v80: 161 pgs: 161 active+clean; 584 KiB data, 54 MiB used, 715 GiB / 715 GiB avail; 114 KiB/s rd, 2.5 KiB/s wr, 240 op/s 2024-03-24T11:16:52.210 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:51 smithi062 ceph-mon[24799]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "foo","app": "rbd"}]': finished 2024-03-24T11:16:52.210 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:51 smithi062 ceph-mon[24799]: osdmap e75: 8 total, 8 up, 8 in 2024-03-24T11:16:52.210 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:51 smithi062 ceph-mon[24799]: pgmap v80: 161 pgs: 161 active+clean; 584 KiB data, 54 MiB used, 715 GiB / 715 GiB avail; 114 KiB/s rd, 2.5 KiB/s wr, 240 op/s 2024-03-24T11:16:52.210 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:51 smithi062 ceph-mon[29666]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "foo","app": "rbd"}]': finished 2024-03-24T11:16:52.210 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:51 smithi062 ceph-mon[29666]: osdmap e75: 8 total, 8 up, 8 in 2024-03-24T11:16:52.210 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:51 smithi062 ceph-mon[29666]: pgmap v80: 161 pgs: 161 active+clean; 584 KiB data, 54 MiB used, 715 GiB / 715 GiB avail; 114 KiB/s rd, 2.5 KiB/s wr, 240 op/s 2024-03-24T11:16:53.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:52 smithi139 ceph-mon[25719]: osdmap e76: 8 total, 8 up, 8 in 2024-03-24T11:16:53.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:52 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:53.214 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:52 smithi062 ceph-mon[29666]: osdmap e76: 8 total, 8 up, 8 in 2024-03-24T11:16:53.215 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:52 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:53.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:52 smithi062 ceph-mon[24799]: osdmap e76: 8 total, 8 up, 8 in 2024-03-24T11:16:53.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:52 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:53.472 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:16:53 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.139 - - [24/Mar/2024:11:16:53] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:16:53.727 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:16:53 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.62 - - [24/Mar/2024:11:16:53] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:16:53.727 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:16:53 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=error ts=2024-03-24T11:16:53.507Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://172.21.15.62:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.62 because it doesn't contain any IP SANs; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://172.21.15.139:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.139 because it doesn't contain any IP SANs" 2024-03-24T11:16:53.727 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:16:53 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:16:53.509Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.139:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.139 because it doesn't contain any IP SANs" 2024-03-24T11:16:53.727 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:16:53 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:16:53.509Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.62:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.62 because it doesn't contain any IP SANs" 2024-03-24T11:16:54.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:53 smithi139 ceph-mon[25719]: osdmap e77: 8 total, 8 up, 8 in 2024-03-24T11:16:54.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:53 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:54.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:53 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:54.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:53 smithi139 ceph-mon[25719]: pgmap v83: 161 pgs: 161 active+clean; 585 KiB data, 59 MiB used, 715 GiB / 715 GiB avail; 66 KiB/s rd, 840 B/s wr, 97 op/s 2024-03-24T11:16:54.128 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:53 smithi062 ceph-mon[24799]: osdmap e77: 8 total, 8 up, 8 in 2024-03-24T11:16:54.128 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:53 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:54.128 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:53 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:54.128 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:53 smithi062 ceph-mon[24799]: pgmap v83: 161 pgs: 161 active+clean; 585 KiB data, 59 MiB used, 715 GiB / 715 GiB avail; 66 KiB/s rd, 840 B/s wr, 97 op/s 2024-03-24T11:16:54.128 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:53 smithi062 ceph-mon[29666]: osdmap e77: 8 total, 8 up, 8 in 2024-03-24T11:16:54.128 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:53 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:54.128 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:53 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:54.128 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:53 smithi062 ceph-mon[29666]: pgmap v83: 161 pgs: 161 active+clean; 585 KiB data, 59 MiB used, 715 GiB / 715 GiB avail; 66 KiB/s rd, 840 B/s wr, 97 op/s 2024-03-24T11:16:54.156 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph orch apply iscsi foo u p' 2024-03-24T11:16:55.177 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:16:54 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.62 - - [24/Mar/2024:11:16:54] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:16:55.706 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:16:55 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.62 - - [24/Mar/2024:11:16:55] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:16:55.977 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:16:55 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.139 - - [24/Mar/2024:11:16:55] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:16:55.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:55 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:55.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:55 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:55.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:55 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:55.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:55 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:55.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:55 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:55.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:55 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:16:55.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:55 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:16:55.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:55 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:55.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:55 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:55.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:55 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi062.xvpxnk", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-03-24T11:16:55.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:55 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi062.xvpxnk", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-03-24T11:16:55.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:55 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:55.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:55 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:16:55.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:55 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:55.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:55 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:55.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:55 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:55.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:55 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:55.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:55 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:55.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:55 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:16:55.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:55 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:16:55.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:55 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:55.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:55 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:55.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:55 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi062.xvpxnk", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-03-24T11:16:55.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:55 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi062.xvpxnk", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-03-24T11:16:55.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:55 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:55.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:55 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:16:56.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:55 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:56.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:55 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:56.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:55 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:56.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:55 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:56.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:55 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:56.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:55 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:16:56.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:55 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:16:56.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:55 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:56.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:55 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:56.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:55 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi062.xvpxnk", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-03-24T11:16:56.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:55 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi062.xvpxnk", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-03-24T11:16:56.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:55 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:56.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:55 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:16:56.890 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:16:56 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.62 - - [24/Mar/2024:11:16:56] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:16:57.186 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:56 smithi062 ceph-mon[24799]: pgmap v84: 161 pgs: 161 active+clean; 585 KiB data, 60 MiB used, 715 GiB / 715 GiB avail; 102 KiB/s rd, 682 B/s wr, 153 op/s 2024-03-24T11:16:57.186 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:56 smithi062 ceph-mon[24799]: Saving service rgw.smpl spec with placement count:2 2024-03-24T11:16:57.186 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:56 smithi062 ceph-mon[24799]: Deploying daemon rgw.smpl.smithi062.xvpxnk on smithi062 2024-03-24T11:16:57.186 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:56 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:57.187 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:56 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:57.187 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:56 smithi062 ceph-mon[29666]: pgmap v84: 161 pgs: 161 active+clean; 585 KiB data, 60 MiB used, 715 GiB / 715 GiB avail; 102 KiB/s rd, 682 B/s wr, 153 op/s 2024-03-24T11:16:57.187 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:56 smithi062 ceph-mon[29666]: Saving service rgw.smpl spec with placement count:2 2024-03-24T11:16:57.187 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:56 smithi062 ceph-mon[29666]: Deploying daemon rgw.smpl.smithi062.xvpxnk on smithi062 2024-03-24T11:16:57.187 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:56 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:57.187 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:56 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:57.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:56 smithi139 ceph-mon[25719]: pgmap v84: 161 pgs: 161 active+clean; 585 KiB data, 60 MiB used, 715 GiB / 715 GiB avail; 102 KiB/s rd, 682 B/s wr, 153 op/s 2024-03-24T11:16:57.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:56 smithi139 ceph-mon[25719]: Saving service rgw.smpl spec with placement count:2 2024-03-24T11:16:57.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:56 smithi139 ceph-mon[25719]: Deploying daemon rgw.smpl.smithi062.xvpxnk on smithi062 2024-03-24T11:16:57.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:56 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:57.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:56 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:57.718 INFO:teuthology.orchestra.run.smithi062.stdout:Scheduled iscsi.foo update... 2024-03-24T11:16:58.448 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'sleep 120' 2024-03-24T11:16:58.976 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:16:58 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:16:58] "GET /metrics HTTP/1.1" 200 203616 "" "Prometheus/2.33.4" 2024-03-24T11:16:58.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:58 smithi062 ceph-mon[29666]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "foo", "api_user": "u", "api_password": "p", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:16:58.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:58 smithi062 ceph-mon[29666]: Saving service iscsi.foo spec with placement count:1 2024-03-24T11:16:58.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:58 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:58.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:58 smithi062 ceph-mon[29666]: pgmap v85: 161 pgs: 161 active+clean; 585 KiB data, 60 MiB used, 715 GiB / 715 GiB avail; 88 KiB/s rd, 591 B/s wr, 133 op/s 2024-03-24T11:16:58.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:58 smithi062 ceph-mon[29666]: osdmap e78: 8 total, 8 up, 8 in 2024-03-24T11:16:58.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:16:58 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:58.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:58 smithi062 ceph-mon[24799]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "foo", "api_user": "u", "api_password": "p", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:16:58.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:58 smithi062 ceph-mon[24799]: Saving service iscsi.foo spec with placement count:1 2024-03-24T11:16:58.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:58 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:58.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:58 smithi062 ceph-mon[24799]: pgmap v85: 161 pgs: 161 active+clean; 585 KiB data, 60 MiB used, 715 GiB / 715 GiB avail; 88 KiB/s rd, 591 B/s wr, 133 op/s 2024-03-24T11:16:58.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:58 smithi062 ceph-mon[24799]: osdmap e78: 8 total, 8 up, 8 in 2024-03-24T11:16:58.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:16:58 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:59.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:58 smithi139 ceph-mon[25719]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "foo", "api_user": "u", "api_password": "p", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:16:59.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:58 smithi139 ceph-mon[25719]: Saving service iscsi.foo spec with placement count:1 2024-03-24T11:16:59.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:58 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:16:59.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:58 smithi139 ceph-mon[25719]: pgmap v85: 161 pgs: 161 active+clean; 585 KiB data, 60 MiB used, 715 GiB / 715 GiB avail; 88 KiB/s rd, 591 B/s wr, 133 op/s 2024-03-24T11:16:59.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:58 smithi139 ceph-mon[25719]: osdmap e78: 8 total, 8 up, 8 in 2024-03-24T11:16:59.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:16:58 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:00.946 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:00 smithi062 ceph-mon[24799]: pgmap v87: 161 pgs: 161 active+clean; 585 KiB data, 61 MiB used, 715 GiB / 715 GiB avail; 77 KiB/s rd, 518 B/s wr, 116 op/s 2024-03-24T11:17:00.947 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:00 smithi062 ceph-mon[29666]: pgmap v87: 161 pgs: 161 active+clean; 585 KiB data, 61 MiB used, 715 GiB / 715 GiB avail; 77 KiB/s rd, 518 B/s wr, 116 op/s 2024-03-24T11:17:01.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:00 smithi139 ceph-mon[25719]: pgmap v87: 161 pgs: 161 active+clean; 585 KiB data, 61 MiB used, 715 GiB / 715 GiB avail; 77 KiB/s rd, 518 B/s wr, 116 op/s 2024-03-24T11:17:01.108 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:17:00 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:17:00] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-03-24T11:17:02.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:02 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:02.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:02 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi139.pbtrmp", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-03-24T11:17:02.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:02 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi139.pbtrmp", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-03-24T11:17:02.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:02 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:02.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:02 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:17:02.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:02 smithi139 ceph-mon[25719]: Deploying daemon rgw.smpl.smithi139.pbtrmp on smithi139 2024-03-24T11:17:02.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:02 smithi139 ceph-mon[25719]: pgmap v88: 161 pgs: 161 active+clean; 585 KiB data, 61 MiB used, 715 GiB / 715 GiB avail; 69 KiB/s rd, 460 B/s wr, 103 op/s 2024-03-24T11:17:03.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:02 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:03.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:02 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi139.pbtrmp", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-03-24T11:17:03.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:02 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi139.pbtrmp", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-03-24T11:17:03.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:02 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:03.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:02 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:17:03.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:02 smithi062 ceph-mon[24799]: Deploying daemon rgw.smpl.smithi139.pbtrmp on smithi139 2024-03-24T11:17:03.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:02 smithi062 ceph-mon[24799]: pgmap v88: 161 pgs: 161 active+clean; 585 KiB data, 61 MiB used, 715 GiB / 715 GiB avail; 69 KiB/s rd, 460 B/s wr, 103 op/s 2024-03-24T11:17:03.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:02 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:03.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:02 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi139.pbtrmp", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-03-24T11:17:03.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:02 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi139.pbtrmp", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-03-24T11:17:03.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:02 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:03.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:02 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:17:03.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:02 smithi062 ceph-mon[29666]: Deploying daemon rgw.smpl.smithi139.pbtrmp on smithi139 2024-03-24T11:17:03.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:02 smithi062 ceph-mon[29666]: pgmap v88: 161 pgs: 161 active+clean; 585 KiB data, 61 MiB used, 715 GiB / 715 GiB avail; 69 KiB/s rd, 460 B/s wr, 103 op/s 2024-03-24T11:17:03.976 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:17:03 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=error ts=2024-03-24T11:17:03.508Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://172.21.15.139:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.139 because it doesn't contain any IP SANs; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://172.21.15.62:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.62 because it doesn't contain any IP SANs" 2024-03-24T11:17:03.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:17:03 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:17:03.510Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.139:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.139 because it doesn't contain any IP SANs" 2024-03-24T11:17:03.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:17:03 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:17:03.510Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.62:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.62 because it doesn't contain any IP SANs" 2024-03-24T11:17:04.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:04 smithi062 ceph-mon[24799]: pgmap v89: 161 pgs: 161 active+clean; 585 KiB data, 74 MiB used, 715 GiB / 715 GiB avail; 94 KiB/s rd, 102 B/s wr, 145 op/s 2024-03-24T11:17:04.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:04 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:04.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:04 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:17:04.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:04 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:17:04.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:04 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:04.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:04 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:17:04.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:04 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:17:04.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:04 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:04.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:04 smithi062 ceph-mon[29666]: pgmap v89: 161 pgs: 161 active+clean; 585 KiB data, 74 MiB used, 715 GiB / 715 GiB avail; 94 KiB/s rd, 102 B/s wr, 145 op/s 2024-03-24T11:17:04.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:04 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:04.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:04 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:17:04.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:04 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:17:04.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:04 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:04.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:04 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:17:04.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:04 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:17:04.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:04 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:05.159 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:04 smithi139 ceph-mon[25719]: pgmap v89: 161 pgs: 161 active+clean; 585 KiB data, 74 MiB used, 715 GiB / 715 GiB avail; 94 KiB/s rd, 102 B/s wr, 145 op/s 2024-03-24T11:17:05.159 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:04 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:05.159 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:04 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:17:05.159 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:04 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:17:05.159 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:04 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:05.159 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:04 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:17:05.159 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:04 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:17:05.159 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:04 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:06.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:05 smithi139 ceph-mon[25719]: Checking dashboard <-> RGW credentials 2024-03-24T11:17:06.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:05 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:17:06.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:05 smithi139 ceph-mon[25719]: Checking dashboard <-> RGW credentials 2024-03-24T11:17:06.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:05 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:17:06.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:05 smithi062 ceph-mon[24799]: Checking dashboard <-> RGW credentials 2024-03-24T11:17:06.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:05 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:17:06.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:05 smithi062 ceph-mon[24799]: Checking dashboard <-> RGW credentials 2024-03-24T11:17:06.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:05 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:17:06.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:05 smithi062 ceph-mon[29666]: Checking dashboard <-> RGW credentials 2024-03-24T11:17:06.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:05 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:17:06.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:05 smithi062 ceph-mon[29666]: Checking dashboard <-> RGW credentials 2024-03-24T11:17:06.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:05 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:17:06.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:06 smithi062 ceph-mon[24799]: pgmap v90: 161 pgs: 161 active+clean; 585 KiB data, 74 MiB used, 715 GiB / 715 GiB avail; 65 KiB/s rd, 102 B/s wr, 100 op/s 2024-03-24T11:17:06.965 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:06 smithi062 ceph-mon[29666]: pgmap v90: 161 pgs: 161 active+clean; 585 KiB data, 74 MiB used, 715 GiB / 715 GiB avail; 65 KiB/s rd, 102 B/s wr, 100 op/s 2024-03-24T11:17:07.231 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:06 smithi139 ceph-mon[25719]: pgmap v90: 161 pgs: 161 active+clean; 585 KiB data, 74 MiB used, 715 GiB / 715 GiB avail; 65 KiB/s rd, 102 B/s wr, 100 op/s 2024-03-24T11:17:07.977 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:17:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.62 - - [24/Mar/2024:11:17:07] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:17:08.269 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:17:08 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.139 - - [24/Mar/2024:11:17:08] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:17:08.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:08 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:08.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:08 smithi062 ceph-mon[24799]: pgmap v91: 161 pgs: 161 active+clean; 585 KiB data, 74 MiB used, 715 GiB / 715 GiB avail; 65 KiB/s rd, 102 B/s wr, 100 op/s 2024-03-24T11:17:08.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:08 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:08.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:08 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:08.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:08 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:08.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:08 smithi062 ceph-mon[29666]: pgmap v91: 161 pgs: 161 active+clean; 585 KiB data, 74 MiB used, 715 GiB / 715 GiB avail; 65 KiB/s rd, 102 B/s wr, 100 op/s 2024-03-24T11:17:08.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:08 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:08.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:08 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:08.978 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:17:08 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:17:08] "GET /metrics HTTP/1.1" 200 210716 "" "Prometheus/2.33.4" 2024-03-24T11:17:09.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:08 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:09.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:08 smithi139 ceph-mon[25719]: pgmap v91: 161 pgs: 161 active+clean; 585 KiB data, 74 MiB used, 715 GiB / 715 GiB avail; 65 KiB/s rd, 102 B/s wr, 100 op/s 2024-03-24T11:17:09.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:08 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:09.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:08 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:09.977 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:17:09 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.62 - - [24/Mar/2024:11:17:09] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:17:10.857 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:17:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:17:10] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-03-24T11:17:10.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:10 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:10.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:10 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:10.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:10 smithi139 ceph-mon[25719]: pgmap v92: 161 pgs: 161 active+clean; 585 KiB data, 91 MiB used, 715 GiB / 715 GiB avail; 114 KiB/s rd, 170 B/s wr, 173 op/s 2024-03-24T11:17:10.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:10 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:10.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:10 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:10.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:10 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:17:10.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:10 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:17:10.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:10 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:10.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:10 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi062.dggsvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-03-24T11:17:10.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:10 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi062.dggsvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-03-24T11:17:10.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:10 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:17:10.976 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:17:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.139 - - [24/Mar/2024:11:17:10] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:17:10.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:10 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:10.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:10 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:10.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:10 smithi062 ceph-mon[24799]: pgmap v92: 161 pgs: 161 active+clean; 585 KiB data, 91 MiB used, 715 GiB / 715 GiB avail; 114 KiB/s rd, 170 B/s wr, 173 op/s 2024-03-24T11:17:10.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:10 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:10.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:10 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:10.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:10 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:17:10.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:10 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:17:10.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:10 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:10.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:10 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi062.dggsvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-03-24T11:17:10.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:10 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi062.dggsvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-03-24T11:17:10.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:10 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:17:10.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:10 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:10.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:10 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:10.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:10 smithi062 ceph-mon[29666]: pgmap v92: 161 pgs: 161 active+clean; 585 KiB data, 91 MiB used, 715 GiB / 715 GiB avail; 114 KiB/s rd, 170 B/s wr, 173 op/s 2024-03-24T11:17:10.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:10 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:10.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:10 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:10.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:10 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:17:10.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:10 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:17:10.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:10 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:10.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:10 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi062.dggsvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-03-24T11:17:10.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:10 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi062.dggsvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-03-24T11:17:10.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:10 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:17:11.976 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:11 smithi062 ceph-mon[24799]: Checking pool "foo" exists for service iscsi.foo 2024-03-24T11:17:11.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:11 smithi062 ceph-mon[24799]: Deploying daemon iscsi.foo.smithi062.dggsvk on smithi062 2024-03-24T11:17:11.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:11 smithi062 ceph-mon[29666]: Checking pool "foo" exists for service iscsi.foo 2024-03-24T11:17:11.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:11 smithi062 ceph-mon[29666]: Deploying daemon iscsi.foo.smithi062.dggsvk on smithi062 2024-03-24T11:17:12.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:11 smithi139 ceph-mon[25719]: Checking pool "foo" exists for service iscsi.foo 2024-03-24T11:17:12.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:11 smithi139 ceph-mon[25719]: Deploying daemon iscsi.foo.smithi062.dggsvk on smithi062 2024-03-24T11:17:12.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:12 smithi062 ceph-mon[24799]: mgrmap e21: y(active, since 2m), standbys: x 2024-03-24T11:17:12.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:12 smithi062 ceph-mon[24799]: pgmap v93: 161 pgs: 161 active+clean; 585 KiB data, 91 MiB used, 715 GiB / 715 GiB avail; 114 KiB/s rd, 170 B/s wr, 173 op/s 2024-03-24T11:17:12.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:12 smithi062 ceph-mon[29666]: mgrmap e21: y(active, since 2m), standbys: x 2024-03-24T11:17:12.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:12 smithi062 ceph-mon[29666]: pgmap v93: 161 pgs: 161 active+clean; 585 KiB data, 91 MiB used, 715 GiB / 715 GiB avail; 114 KiB/s rd, 170 B/s wr, 173 op/s 2024-03-24T11:17:13.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:12 smithi139 ceph-mon[25719]: mgrmap e21: y(active, since 2m), standbys: x 2024-03-24T11:17:13.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:12 smithi139 ceph-mon[25719]: pgmap v93: 161 pgs: 161 active+clean; 585 KiB data, 91 MiB used, 715 GiB / 715 GiB avail; 114 KiB/s rd, 170 B/s wr, 173 op/s 2024-03-24T11:17:13.726 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:17:13 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=error ts=2024-03-24T11:17:13.508Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://172.21.15.139:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.139 because it doesn't contain any IP SANs; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://172.21.15.62:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.62 because it doesn't contain any IP SANs" 2024-03-24T11:17:13.727 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:17:13 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:17:13.510Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.62:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.62 because it doesn't contain any IP SANs" 2024-03-24T11:17:13.727 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:17:13 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:17:13.510Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.139:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.139 because it doesn't contain any IP SANs" 2024-03-24T11:17:15.186 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:14 smithi062 ceph-mon[29666]: pgmap v94: 161 pgs: 161 active+clean; 585 KiB data, 91 MiB used, 715 GiB / 715 GiB avail; 114 KiB/s rd, 170 B/s wr, 173 op/s 2024-03-24T11:17:15.186 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:14 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:15.186 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:14 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:17:15.186 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:14 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:17:15.186 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:14 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:15.186 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:14 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-03-24T11:17:15.186 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:14 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-03-24T11:17:15.186 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:14 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:15.186 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:14 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi062"}]: dispatch 2024-03-24T11:17:15.187 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:14 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:15.187 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:14 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:17:15.187 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:14 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:17:15.187 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:14 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:15.187 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:14 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/2244316681' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-24T11:17:15.187 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:14 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/453321562' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6801/621428596"}]: dispatch 2024-03-24T11:17:15.187 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:14 smithi062 ceph-mon[24799]: pgmap v94: 161 pgs: 161 active+clean; 585 KiB data, 91 MiB used, 715 GiB / 715 GiB avail; 114 KiB/s rd, 170 B/s wr, 173 op/s 2024-03-24T11:17:15.187 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:14 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:15.187 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:14 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:17:15.187 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:14 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:17:15.187 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:14 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:15.187 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:14 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-03-24T11:17:15.188 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:14 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-03-24T11:17:15.188 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:14 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:15.188 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:14 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi062"}]: dispatch 2024-03-24T11:17:15.188 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:14 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:15.188 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:14 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:17:15.188 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:14 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:17:15.188 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:14 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:15.188 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:14 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/2244316681' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-24T11:17:15.188 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:14 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/453321562' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6801/621428596"}]: dispatch 2024-03-24T11:17:15.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:14 smithi139 ceph-mon[25719]: pgmap v94: 161 pgs: 161 active+clean; 585 KiB data, 91 MiB used, 715 GiB / 715 GiB avail; 114 KiB/s rd, 170 B/s wr, 173 op/s 2024-03-24T11:17:15.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:14 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:15.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:14 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:17:15.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:14 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:17:15.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:14 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:15.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:14 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-03-24T11:17:15.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:14 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-03-24T11:17:15.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:14 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:15.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:14 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi062"}]: dispatch 2024-03-24T11:17:15.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:14 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:15.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:14 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:17:15.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:14 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:17:15.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:14 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:15.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:14 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/2244316681' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-24T11:17:15.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:14 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/453321562' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6801/621428596"}]: dispatch 2024-03-24T11:17:16.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:15 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:17:16.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:15 smithi139 ceph-mon[25719]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-03-24T11:17:16.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:15 smithi139 ceph-mon[25719]: Adding iSCSI gateway http://:@172.21.15.62:5000 to Dashboard 2024-03-24T11:17:16.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:15 smithi139 ceph-mon[25719]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-03-24T11:17:16.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:15 smithi139 ceph-mon[25719]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi062"}]: dispatch 2024-03-24T11:17:16.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:15 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:17:16.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:15 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/453321562' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6801/621428596"}]': finished 2024-03-24T11:17:16.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:15 smithi139 ceph-mon[25719]: osdmap e79: 8 total, 8 up, 8 in 2024-03-24T11:17:16.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:15 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/2241890065' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6800/621428596"}]: dispatch 2024-03-24T11:17:16.182 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:15 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:17:16.183 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:15 smithi062 ceph-mon[24799]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-03-24T11:17:16.183 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:15 smithi062 ceph-mon[24799]: Adding iSCSI gateway http://:@172.21.15.62:5000 to Dashboard 2024-03-24T11:17:16.183 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:15 smithi062 ceph-mon[24799]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-03-24T11:17:16.183 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:15 smithi062 ceph-mon[24799]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi062"}]: dispatch 2024-03-24T11:17:16.183 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:15 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:17:16.183 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:15 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/453321562' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6801/621428596"}]': finished 2024-03-24T11:17:16.183 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:15 smithi062 ceph-mon[24799]: osdmap e79: 8 total, 8 up, 8 in 2024-03-24T11:17:16.183 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:15 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/2241890065' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6800/621428596"}]: dispatch 2024-03-24T11:17:16.183 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:15 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:17:16.183 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:15 smithi062 ceph-mon[29666]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-03-24T11:17:16.184 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:15 smithi062 ceph-mon[29666]: Adding iSCSI gateway http://:@172.21.15.62:5000 to Dashboard 2024-03-24T11:17:16.184 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:15 smithi062 ceph-mon[29666]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-03-24T11:17:16.184 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:15 smithi062 ceph-mon[29666]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi062"}]: dispatch 2024-03-24T11:17:16.184 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:15 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:17:16.184 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:15 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/453321562' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6801/621428596"}]': finished 2024-03-24T11:17:16.184 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:15 smithi062 ceph-mon[29666]: osdmap e79: 8 total, 8 up, 8 in 2024-03-24T11:17:16.184 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:15 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/2241890065' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6800/621428596"}]: dispatch 2024-03-24T11:17:17.183 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:16 smithi062 ceph-mon[24799]: pgmap v96: 161 pgs: 161 active+clean; 585 KiB data, 91 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s rd, 102 B/s wr, 107 op/s 2024-03-24T11:17:17.183 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:16 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/2241890065' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6800/621428596"}]': finished 2024-03-24T11:17:17.183 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:16 smithi062 ceph-mon[24799]: osdmap e80: 8 total, 8 up, 8 in 2024-03-24T11:17:17.183 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:16 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/3812836557' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/1654711111"}]: dispatch 2024-03-24T11:17:17.184 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:16 smithi062 ceph-mon[29666]: pgmap v96: 161 pgs: 161 active+clean; 585 KiB data, 91 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s rd, 102 B/s wr, 107 op/s 2024-03-24T11:17:17.184 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:16 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/2241890065' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6800/621428596"}]': finished 2024-03-24T11:17:17.184 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:16 smithi062 ceph-mon[29666]: osdmap e80: 8 total, 8 up, 8 in 2024-03-24T11:17:17.184 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:16 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/3812836557' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/1654711111"}]: dispatch 2024-03-24T11:17:17.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:16 smithi139 ceph-mon[25719]: pgmap v96: 161 pgs: 161 active+clean; 585 KiB data, 91 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s rd, 102 B/s wr, 107 op/s 2024-03-24T11:17:17.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:16 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/2241890065' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6800/621428596"}]': finished 2024-03-24T11:17:17.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:16 smithi139 ceph-mon[25719]: osdmap e80: 8 total, 8 up, 8 in 2024-03-24T11:17:17.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:16 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/3812836557' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/1654711111"}]: dispatch 2024-03-24T11:17:18.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:18 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/3812836557' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/1654711111"}]': finished 2024-03-24T11:17:18.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:18 smithi062 ceph-mon[24799]: osdmap e81: 8 total, 8 up, 8 in 2024-03-24T11:17:18.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:18 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/2761839079' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6801/2609891207"}]: dispatch 2024-03-24T11:17:18.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:18 smithi062 ceph-mon[24799]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6801/2609891207"}]: dispatch 2024-03-24T11:17:18.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:18 smithi062 ceph-mon[24799]: pgmap v99: 161 pgs: 161 active+clean; 585 KiB data, 91 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:17:18.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:18 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:18.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:18 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/3812836557' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/1654711111"}]': finished 2024-03-24T11:17:18.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:18 smithi062 ceph-mon[29666]: osdmap e81: 8 total, 8 up, 8 in 2024-03-24T11:17:18.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:18 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/2761839079' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6801/2609891207"}]: dispatch 2024-03-24T11:17:18.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:18 smithi062 ceph-mon[29666]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6801/2609891207"}]: dispatch 2024-03-24T11:17:18.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:18 smithi062 ceph-mon[29666]: pgmap v99: 161 pgs: 161 active+clean; 585 KiB data, 91 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:17:18.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:18 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:18.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:18 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/3812836557' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/1654711111"}]': finished 2024-03-24T11:17:18.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:18 smithi139 ceph-mon[25719]: osdmap e81: 8 total, 8 up, 8 in 2024-03-24T11:17:18.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:18 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/2761839079' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6801/2609891207"}]: dispatch 2024-03-24T11:17:18.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:18 smithi139 ceph-mon[25719]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6801/2609891207"}]: dispatch 2024-03-24T11:17:18.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:18 smithi139 ceph-mon[25719]: pgmap v99: 161 pgs: 161 active+clean; 585 KiB data, 91 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:17:18.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:18 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:18.888 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:17:18 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.62 - - [24/Mar/2024:11:17:18] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:17:19.175 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:17:18 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:17:18] "GET /metrics HTTP/1.1" 200 214878 "" "Prometheus/2.33.4" 2024-03-24T11:17:19.442 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:19 smithi062 ceph-mon[24799]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6801/2609891207"}]': finished 2024-03-24T11:17:19.442 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:19 smithi062 ceph-mon[24799]: osdmap e82: 8 total, 8 up, 8 in 2024-03-24T11:17:19.442 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:19 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/4186397503' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/2764833578"}]: dispatch 2024-03-24T11:17:19.442 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:19 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:19.443 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:19 smithi062 ceph-mon[29666]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6801/2609891207"}]': finished 2024-03-24T11:17:19.443 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:19 smithi062 ceph-mon[29666]: osdmap e82: 8 total, 8 up, 8 in 2024-03-24T11:17:19.443 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:19 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/4186397503' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/2764833578"}]: dispatch 2024-03-24T11:17:19.443 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:19 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:19.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:19 smithi139 ceph-mon[25719]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6801/2609891207"}]': finished 2024-03-24T11:17:19.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:19 smithi139 ceph-mon[25719]: osdmap e82: 8 total, 8 up, 8 in 2024-03-24T11:17:19.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:19 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/4186397503' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/2764833578"}]: dispatch 2024-03-24T11:17:19.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:19 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:20.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:20 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/4186397503' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/2764833578"}]': finished 2024-03-24T11:17:20.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:20 smithi062 ceph-mon[24799]: osdmap e83: 8 total, 8 up, 8 in 2024-03-24T11:17:20.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:20 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/1132499108' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6800/2609891207"}]: dispatch 2024-03-24T11:17:20.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:20 smithi062 ceph-mon[24799]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6800/2609891207"}]: dispatch 2024-03-24T11:17:20.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:20 smithi062 ceph-mon[24799]: pgmap v102: 161 pgs: 161 active+clean; 585 KiB data, 92 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 3 op/s 2024-03-24T11:17:20.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:20 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/4186397503' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/2764833578"}]': finished 2024-03-24T11:17:20.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:20 smithi062 ceph-mon[29666]: osdmap e83: 8 total, 8 up, 8 in 2024-03-24T11:17:20.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:20 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/1132499108' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6800/2609891207"}]: dispatch 2024-03-24T11:17:20.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:20 smithi062 ceph-mon[29666]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6800/2609891207"}]: dispatch 2024-03-24T11:17:20.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:20 smithi062 ceph-mon[29666]: pgmap v102: 161 pgs: 161 active+clean; 585 KiB data, 92 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 3 op/s 2024-03-24T11:17:20.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:20 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/4186397503' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/2764833578"}]': finished 2024-03-24T11:17:20.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:20 smithi139 ceph-mon[25719]: osdmap e83: 8 total, 8 up, 8 in 2024-03-24T11:17:20.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:20 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/1132499108' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6800/2609891207"}]: dispatch 2024-03-24T11:17:20.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:20 smithi139 ceph-mon[25719]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6800/2609891207"}]: dispatch 2024-03-24T11:17:20.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:20 smithi139 ceph-mon[25719]: pgmap v102: 161 pgs: 161 active+clean; 585 KiB data, 92 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 3 op/s 2024-03-24T11:17:21.108 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:17:20 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:17:20] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-03-24T11:17:21.476 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:17:21 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.62 - - [24/Mar/2024:11:17:21] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:17:21.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:21 smithi062 ceph-mon[24799]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6800/2609891207"}]': finished 2024-03-24T11:17:21.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:21 smithi062 ceph-mon[24799]: osdmap e84: 8 total, 8 up, 8 in 2024-03-24T11:17:21.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:21 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/2278964922' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/81242112"}]: dispatch 2024-03-24T11:17:21.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:21 smithi062 ceph-mon[29666]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6800/2609891207"}]': finished 2024-03-24T11:17:21.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:21 smithi062 ceph-mon[29666]: osdmap e84: 8 total, 8 up, 8 in 2024-03-24T11:17:21.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:21 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/2278964922' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/81242112"}]: dispatch 2024-03-24T11:17:21.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:21 smithi139 ceph-mon[25719]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6800/2609891207"}]': finished 2024-03-24T11:17:21.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:21 smithi139 ceph-mon[25719]: osdmap e84: 8 total, 8 up, 8 in 2024-03-24T11:17:21.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:21 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/2278964922' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/81242112"}]: dispatch 2024-03-24T11:17:22.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:22 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/2278964922' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/81242112"}]': finished 2024-03-24T11:17:22.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:22 smithi062 ceph-mon[24799]: osdmap e85: 8 total, 8 up, 8 in 2024-03-24T11:17:22.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:22 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:22.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:22 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:22.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:22 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:17:22.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:22 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:17:22.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:22 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:22.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:22 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/627247422' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/2403755398"}]: dispatch 2024-03-24T11:17:22.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:22 smithi062 ceph-mon[24799]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/2403755398"}]: dispatch 2024-03-24T11:17:22.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:22 smithi062 ceph-mon[24799]: pgmap v105: 161 pgs: 161 active+clean; 585 KiB data, 92 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 3 op/s 2024-03-24T11:17:22.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:22 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/2278964922' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/81242112"}]': finished 2024-03-24T11:17:22.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:22 smithi062 ceph-mon[29666]: osdmap e85: 8 total, 8 up, 8 in 2024-03-24T11:17:22.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:22 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:22.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:22 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:22.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:22 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:17:22.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:22 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:17:22.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:22 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:22.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:22 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/627247422' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/2403755398"}]: dispatch 2024-03-24T11:17:22.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:22 smithi062 ceph-mon[29666]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/2403755398"}]: dispatch 2024-03-24T11:17:22.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:22 smithi062 ceph-mon[29666]: pgmap v105: 161 pgs: 161 active+clean; 585 KiB data, 92 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 3 op/s 2024-03-24T11:17:22.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:22 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/2278964922' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/81242112"}]': finished 2024-03-24T11:17:22.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:22 smithi139 ceph-mon[25719]: osdmap e85: 8 total, 8 up, 8 in 2024-03-24T11:17:22.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:22 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:22.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:22 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:22.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:22 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:17:22.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:22 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:17:22.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:22 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:22.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:22 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/627247422' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/2403755398"}]: dispatch 2024-03-24T11:17:22.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:22 smithi139 ceph-mon[25719]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/2403755398"}]: dispatch 2024-03-24T11:17:22.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:22 smithi139 ceph-mon[25719]: pgmap v105: 161 pgs: 161 active+clean; 585 KiB data, 92 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 3 op/s 2024-03-24T11:17:23.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:23 smithi139 ceph-mon[25719]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/2403755398"}]': finished 2024-03-24T11:17:23.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:23 smithi139 ceph-mon[25719]: osdmap e86: 8 total, 8 up, 8 in 2024-03-24T11:17:23.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:23 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/1151223693' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/2039307015"}]: dispatch 2024-03-24T11:17:23.726 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:23 smithi062 ceph-mon[24799]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/2403755398"}]': finished 2024-03-24T11:17:23.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:23 smithi062 ceph-mon[24799]: osdmap e86: 8 total, 8 up, 8 in 2024-03-24T11:17:23.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:23 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/1151223693' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/2039307015"}]: dispatch 2024-03-24T11:17:23.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:23 smithi062 ceph-mon[29666]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/2403755398"}]': finished 2024-03-24T11:17:23.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:23 smithi062 ceph-mon[29666]: osdmap e86: 8 total, 8 up, 8 in 2024-03-24T11:17:23.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:23 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/1151223693' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/2039307015"}]: dispatch 2024-03-24T11:17:23.727 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:17:23 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=error ts=2024-03-24T11:17:23.508Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://172.21.15.139:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.139 because it doesn't contain any IP SANs; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://172.21.15.62:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.62 because it doesn't contain any IP SANs" 2024-03-24T11:17:23.727 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:17:23 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:17:23.510Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.62:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.62 because it doesn't contain any IP SANs" 2024-03-24T11:17:23.728 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:17:23 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:17:23.510Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.139:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.139 because it doesn't contain any IP SANs" 2024-03-24T11:17:24.476 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:24 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/1151223693' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/2039307015"}]': finished 2024-03-24T11:17:24.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:24 smithi062 ceph-mon[29666]: osdmap e87: 8 total, 8 up, 8 in 2024-03-24T11:17:24.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:24 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/3419018261' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/4080145397"}]: dispatch 2024-03-24T11:17:24.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:24 smithi062 ceph-mon[29666]: pgmap v108: 161 pgs: 161 active+clean; 585 KiB data, 93 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:17:24.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:24 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/1151223693' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/2039307015"}]': finished 2024-03-24T11:17:24.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:24 smithi062 ceph-mon[24799]: osdmap e87: 8 total, 8 up, 8 in 2024-03-24T11:17:24.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:24 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/3419018261' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/4080145397"}]: dispatch 2024-03-24T11:17:24.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:24 smithi062 ceph-mon[24799]: pgmap v108: 161 pgs: 161 active+clean; 585 KiB data, 93 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:17:24.607 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:24 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/1151223693' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/2039307015"}]': finished 2024-03-24T11:17:24.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:24 smithi139 ceph-mon[25719]: osdmap e87: 8 total, 8 up, 8 in 2024-03-24T11:17:24.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:24 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/3419018261' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/4080145397"}]: dispatch 2024-03-24T11:17:24.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:24 smithi139 ceph-mon[25719]: pgmap v108: 161 pgs: 161 active+clean; 585 KiB data, 93 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:17:25.226 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:17:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.139 - - [24/Mar/2024:11:17:24] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:17:25.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:25 smithi139 ceph-mon[25719]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:17:25.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:25 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/3419018261' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/4080145397"}]': finished 2024-03-24T11:17:25.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:25 smithi139 ceph-mon[25719]: osdmap e88: 8 total, 8 up, 8 in 2024-03-24T11:17:25.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:25 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/1169915228' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/936493210"}]: dispatch 2024-03-24T11:17:25.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:25 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:25.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:25 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:25.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:25 smithi062 ceph-mon[24799]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:17:25.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:25 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/3419018261' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/4080145397"}]': finished 2024-03-24T11:17:25.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:25 smithi062 ceph-mon[24799]: osdmap e88: 8 total, 8 up, 8 in 2024-03-24T11:17:25.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:25 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/1169915228' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/936493210"}]: dispatch 2024-03-24T11:17:25.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:25 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:25.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:25 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:25.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:25 smithi062 ceph-mon[29666]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:17:25.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:25 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/3419018261' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/4080145397"}]': finished 2024-03-24T11:17:25.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:25 smithi062 ceph-mon[29666]: osdmap e88: 8 total, 8 up, 8 in 2024-03-24T11:17:25.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:25 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/1169915228' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/936493210"}]: dispatch 2024-03-24T11:17:25.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:25 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:25.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:25 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:26.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:26 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/1169915228' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/936493210"}]': finished 2024-03-24T11:17:26.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:26 smithi139 ceph-mon[25719]: osdmap e89: 8 total, 8 up, 8 in 2024-03-24T11:17:26.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:26 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/4178914591' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/2103795741"}]: dispatch 2024-03-24T11:17:26.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:26 smithi139 ceph-mon[25719]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/2103795741"}]: dispatch 2024-03-24T11:17:26.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:26 smithi139 ceph-mon[25719]: pgmap v111: 161 pgs: 161 active+clean; 585 KiB data, 93 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:17:26.654 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:26 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/1169915228' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/936493210"}]': finished 2024-03-24T11:17:26.654 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:26 smithi062 ceph-mon[29666]: osdmap e89: 8 total, 8 up, 8 in 2024-03-24T11:17:26.655 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:26 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/4178914591' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/2103795741"}]: dispatch 2024-03-24T11:17:26.655 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:26 smithi062 ceph-mon[29666]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/2103795741"}]: dispatch 2024-03-24T11:17:26.655 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:26 smithi062 ceph-mon[29666]: pgmap v111: 161 pgs: 161 active+clean; 585 KiB data, 93 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:17:26.655 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:26 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/1169915228' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/936493210"}]': finished 2024-03-24T11:17:26.655 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:26 smithi062 ceph-mon[24799]: osdmap e89: 8 total, 8 up, 8 in 2024-03-24T11:17:26.655 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:26 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/4178914591' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/2103795741"}]: dispatch 2024-03-24T11:17:26.655 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:26 smithi062 ceph-mon[24799]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/2103795741"}]: dispatch 2024-03-24T11:17:26.655 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:26 smithi062 ceph-mon[24799]: pgmap v111: 161 pgs: 161 active+clean; 585 KiB data, 93 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:17:27.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:27 smithi139 ceph-mon[25719]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/2103795741"}]': finished 2024-03-24T11:17:27.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:27 smithi139 ceph-mon[25719]: osdmap e90: 8 total, 8 up, 8 in 2024-03-24T11:17:27.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:27 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/709698161' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/106493134"}]: dispatch 2024-03-24T11:17:27.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:27 smithi062 ceph-mon[24799]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/2103795741"}]': finished 2024-03-24T11:17:27.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:27 smithi062 ceph-mon[24799]: osdmap e90: 8 total, 8 up, 8 in 2024-03-24T11:17:27.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:27 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/709698161' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/106493134"}]: dispatch 2024-03-24T11:17:27.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:27 smithi062 ceph-mon[29666]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/2103795741"}]': finished 2024-03-24T11:17:27.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:27 smithi062 ceph-mon[29666]: osdmap e90: 8 total, 8 up, 8 in 2024-03-24T11:17:27.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:27 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/709698161' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/106493134"}]: dispatch 2024-03-24T11:17:28.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:28 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/709698161' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/106493134"}]': finished 2024-03-24T11:17:28.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:28 smithi139 ceph-mon[25719]: osdmap e91: 8 total, 8 up, 8 in 2024-03-24T11:17:28.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:28 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/4240209737' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6800/1616593108"}]: dispatch 2024-03-24T11:17:28.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:28 smithi139 ceph-mon[25719]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6800/1616593108"}]: dispatch 2024-03-24T11:17:28.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:28 smithi139 ceph-mon[25719]: pgmap v114: 161 pgs: 161 active+clean; 585 KiB data, 93 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:17:28.726 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:28 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/709698161' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/106493134"}]': finished 2024-03-24T11:17:28.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:28 smithi062 ceph-mon[24799]: osdmap e91: 8 total, 8 up, 8 in 2024-03-24T11:17:28.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:28 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/4240209737' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6800/1616593108"}]: dispatch 2024-03-24T11:17:28.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:28 smithi062 ceph-mon[24799]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6800/1616593108"}]: dispatch 2024-03-24T11:17:28.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:28 smithi062 ceph-mon[24799]: pgmap v114: 161 pgs: 161 active+clean; 585 KiB data, 93 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:17:28.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:28 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/709698161' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/106493134"}]': finished 2024-03-24T11:17:28.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:28 smithi062 ceph-mon[29666]: osdmap e91: 8 total, 8 up, 8 in 2024-03-24T11:17:28.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:28 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/4240209737' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6800/1616593108"}]: dispatch 2024-03-24T11:17:28.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:28 smithi062 ceph-mon[29666]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6800/1616593108"}]: dispatch 2024-03-24T11:17:28.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:28 smithi062 ceph-mon[29666]: pgmap v114: 161 pgs: 161 active+clean; 585 KiB data, 93 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:17:29.226 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:17:28 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:17:28] "GET /metrics HTTP/1.1" 200 214878 "" "Prometheus/2.33.4" 2024-03-24T11:17:29.607 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:29 smithi139 ceph-mon[25719]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6800/1616593108"}]': finished 2024-03-24T11:17:29.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:29 smithi139 ceph-mon[25719]: osdmap e92: 8 total, 8 up, 8 in 2024-03-24T11:17:29.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:29 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/3714843951' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/2488622750"}]: dispatch 2024-03-24T11:17:29.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:29 smithi062 ceph-mon[24799]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6800/1616593108"}]': finished 2024-03-24T11:17:29.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:29 smithi062 ceph-mon[24799]: osdmap e92: 8 total, 8 up, 8 in 2024-03-24T11:17:29.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:29 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/3714843951' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/2488622750"}]: dispatch 2024-03-24T11:17:29.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:29 smithi062 ceph-mon[29666]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6800/1616593108"}]': finished 2024-03-24T11:17:29.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:29 smithi062 ceph-mon[29666]: osdmap e92: 8 total, 8 up, 8 in 2024-03-24T11:17:29.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:29 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/3714843951' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/2488622750"}]: dispatch 2024-03-24T11:17:30.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:30 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/3714843951' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/2488622750"}]': finished 2024-03-24T11:17:30.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:30 smithi062 ceph-mon[24799]: osdmap e93: 8 total, 8 up, 8 in 2024-03-24T11:17:30.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:30 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/466612129' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6801/1616593108"}]: dispatch 2024-03-24T11:17:30.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:30 smithi062 ceph-mon[24799]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6801/1616593108"}]: dispatch 2024-03-24T11:17:30.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:30 smithi062 ceph-mon[24799]: pgmap v117: 161 pgs: 161 active+clean; 585 KiB data, 93 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:17:30.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:30 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/3714843951' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/2488622750"}]': finished 2024-03-24T11:17:30.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:30 smithi062 ceph-mon[29666]: osdmap e93: 8 total, 8 up, 8 in 2024-03-24T11:17:30.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:30 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/466612129' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6801/1616593108"}]: dispatch 2024-03-24T11:17:30.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:30 smithi062 ceph-mon[29666]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6801/1616593108"}]: dispatch 2024-03-24T11:17:30.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:30 smithi062 ceph-mon[29666]: pgmap v117: 161 pgs: 161 active+clean; 585 KiB data, 93 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:17:30.777 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:30 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/3714843951' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/2488622750"}]': finished 2024-03-24T11:17:30.777 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:30 smithi139 ceph-mon[25719]: osdmap e93: 8 total, 8 up, 8 in 2024-03-24T11:17:30.777 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:30 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/466612129' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6801/1616593108"}]: dispatch 2024-03-24T11:17:30.777 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:30 smithi139 ceph-mon[25719]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6801/1616593108"}]: dispatch 2024-03-24T11:17:30.777 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:30 smithi139 ceph-mon[25719]: pgmap v117: 161 pgs: 161 active+clean; 585 KiB data, 93 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:17:31.108 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:17:30 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:17:30] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-03-24T11:17:31.726 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:31 smithi062 ceph-mon[24799]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6801/1616593108"}]': finished 2024-03-24T11:17:31.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:31 smithi062 ceph-mon[24799]: osdmap e94: 8 total, 8 up, 8 in 2024-03-24T11:17:31.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:31 smithi062 ceph-mon[29666]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6801/1616593108"}]': finished 2024-03-24T11:17:31.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:31 smithi062 ceph-mon[29666]: osdmap e94: 8 total, 8 up, 8 in 2024-03-24T11:17:31.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:31 smithi139 ceph-mon[25719]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6801/1616593108"}]': finished 2024-03-24T11:17:31.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:31 smithi139 ceph-mon[25719]: osdmap e94: 8 total, 8 up, 8 in 2024-03-24T11:17:32.726 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:32 smithi062 ceph-mon[24799]: pgmap v119: 161 pgs: 161 active+clean; 585 KiB data, 93 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-24T11:17:32.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:32 smithi062 ceph-mon[29666]: pgmap v119: 161 pgs: 161 active+clean; 585 KiB data, 93 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-24T11:17:32.857 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:32 smithi139 ceph-mon[25719]: pgmap v119: 161 pgs: 161 active+clean; 585 KiB data, 93 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-24T11:17:33.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:17:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=error ts=2024-03-24T11:17:33.509Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://172.21.15.139:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.139 because it doesn't contain any IP SANs; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://172.21.15.62:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.62 because it doesn't contain any IP SANs" 2024-03-24T11:17:33.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:17:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:17:33.511Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.62:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.62 because it doesn't contain any IP SANs" 2024-03-24T11:17:33.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:17:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:17:33.511Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.139:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.139 because it doesn't contain any IP SANs" 2024-03-24T11:17:35.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:34 smithi062 ceph-mon[24799]: pgmap v120: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-24T11:17:35.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:34 smithi062 ceph-mon[29666]: pgmap v120: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-24T11:17:35.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:34 smithi139 ceph-mon[25719]: pgmap v120: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-24T11:17:35.972 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:35 smithi062 ceph-mon[24799]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:17:35.973 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:35 smithi062 ceph-mon[29666]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:17:36.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:35 smithi139 ceph-mon[25719]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:17:36.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:36 smithi062 ceph-mon[24799]: pgmap v121: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T11:17:36.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:36 smithi062 ceph-mon[29666]: pgmap v121: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T11:17:37.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:36 smithi139 ceph-mon[25719]: pgmap v121: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T11:17:38.888 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:17:38 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.62 - - [24/Mar/2024:11:17:38] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:17:39.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:38 smithi139 ceph-mon[25719]: pgmap v122: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 601 B/s rd, 0 op/s 2024-03-24T11:17:39.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:38 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:39.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:38 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:39.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:38 smithi062 ceph-mon[24799]: pgmap v122: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 601 B/s rd, 0 op/s 2024-03-24T11:17:39.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:38 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:39.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:38 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:39.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:17:38 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:17:38] "GET /metrics HTTP/1.1" 200 214964 "" "Prometheus/2.33.4" 2024-03-24T11:17:39.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:38 smithi062 ceph-mon[29666]: pgmap v122: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 601 B/s rd, 0 op/s 2024-03-24T11:17:39.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:38 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:39.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:38 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:41.108 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:17:40 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:17:40] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-03-24T11:17:41.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:40 smithi139 ceph-mon[25719]: pgmap v123: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T11:17:41.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:40 smithi062 ceph-mon[24799]: pgmap v123: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T11:17:41.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:40 smithi062 ceph-mon[29666]: pgmap v123: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T11:17:43.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:42 smithi062 ceph-mon[24799]: pgmap v124: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 890 B/s rd, 0 op/s 2024-03-24T11:17:43.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:42 smithi062 ceph-mon[29666]: pgmap v124: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 890 B/s rd, 0 op/s 2024-03-24T11:17:43.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:42 smithi139 ceph-mon[25719]: pgmap v124: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 890 B/s rd, 0 op/s 2024-03-24T11:17:43.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:17:43 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=error ts=2024-03-24T11:17:43.509Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://172.21.15.139:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.139 because it doesn't contain any IP SANs; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://172.21.15.62:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.62 because it doesn't contain any IP SANs" 2024-03-24T11:17:43.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:17:43 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:17:43.511Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.62:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.62 because it doesn't contain any IP SANs" 2024-03-24T11:17:43.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:17:43 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:17:43.512Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.139:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.139 because it doesn't contain any IP SANs" 2024-03-24T11:17:45.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:44 smithi062 ceph-mon[24799]: pgmap v125: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:17:45.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:17:44 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.139 - - [24/Mar/2024:11:17:44] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:17:45.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:44 smithi062 ceph-mon[29666]: pgmap v125: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:17:45.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:44 smithi139 ceph-mon[25719]: pgmap v125: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:17:46.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:45 smithi062 ceph-mon[24799]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:17:46.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:45 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:46.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:45 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:46.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:45 smithi062 ceph-mon[29666]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:17:46.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:45 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:46.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:45 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:46.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:45 smithi139 ceph-mon[25719]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:17:46.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:45 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:46.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:45 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:47.039 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:46 smithi139 ceph-mon[25719]: pgmap v126: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:17:47.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:46 smithi062 ceph-mon[24799]: pgmap v126: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:17:47.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:46 smithi062 ceph-mon[29666]: pgmap v126: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:17:48.037 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:47 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]: dispatch 2024-03-24T11:17:48.037 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:47 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-03-24T11:17:48.037 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:47 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2024-03-24T11:17:48.037 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:47 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]: dispatch 2024-03-24T11:17:48.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:47 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]: dispatch 2024-03-24T11:17:48.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:47 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-03-24T11:17:48.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:47 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2024-03-24T11:17:48.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:47 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]: dispatch 2024-03-24T11:17:48.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:47 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]: dispatch 2024-03-24T11:17:48.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:47 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-03-24T11:17:48.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:47 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2024-03-24T11:17:48.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:47 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]: dispatch 2024-03-24T11:17:49.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:48 smithi139 ceph-mon[25719]: pgmap v127: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:17:49.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:48 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]': finished 2024-03-24T11:17:49.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:48 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]': finished 2024-03-24T11:17:49.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:48 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]': finished 2024-03-24T11:17:49.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:48 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]': finished 2024-03-24T11:17:49.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:48 smithi139 ceph-mon[25719]: osdmap e95: 8 total, 8 up, 8 in 2024-03-24T11:17:49.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:48 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-24T11:17:49.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:48 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-24T11:17:49.226 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:17:48 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:17:48] "GET /metrics HTTP/1.1" 200 214889 "" "Prometheus/2.33.4" 2024-03-24T11:17:49.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:48 smithi062 ceph-mon[29666]: pgmap v127: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:17:49.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:48 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]': finished 2024-03-24T11:17:49.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:48 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]': finished 2024-03-24T11:17:49.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:48 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]': finished 2024-03-24T11:17:49.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:48 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]': finished 2024-03-24T11:17:49.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:48 smithi062 ceph-mon[29666]: osdmap e95: 8 total, 8 up, 8 in 2024-03-24T11:17:49.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:48 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-24T11:17:49.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:48 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-24T11:17:49.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:48 smithi062 ceph-mon[24799]: pgmap v127: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:17:49.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:48 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [7, 4]}]': finished 2024-03-24T11:17:49.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:48 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]': finished 2024-03-24T11:17:49.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:48 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]': finished 2024-03-24T11:17:49.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:48 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]': finished 2024-03-24T11:17:49.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:48 smithi062 ceph-mon[24799]: osdmap e95: 8 total, 8 up, 8 in 2024-03-24T11:17:49.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:48 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-24T11:17:49.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:48 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-24T11:17:50.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:49 smithi062 ceph-mon[24799]: osdmap e96: 8 total, 8 up, 8 in 2024-03-24T11:17:50.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:49 smithi062 ceph-mon[29666]: osdmap e96: 8 total, 8 up, 8 in 2024-03-24T11:17:50.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:49 smithi139 ceph-mon[25719]: osdmap e96: 8 total, 8 up, 8 in 2024-03-24T11:17:51.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:50 smithi139 ceph-mon[25719]: pgmap v130: 161 pgs: 4 peering, 157 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:17:51.108 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:17:50 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:17:50] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-03-24T11:17:51.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:50 smithi062 ceph-mon[24799]: pgmap v130: 161 pgs: 4 peering, 157 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:17:51.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:50 smithi062 ceph-mon[29666]: pgmap v130: 161 pgs: 4 peering, 157 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:17:52.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:51 smithi062 ceph-mon[24799]: pgmap v131: 161 pgs: 4 peering, 157 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-24T11:17:52.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:51 smithi062 ceph-mon[29666]: pgmap v131: 161 pgs: 4 peering, 157 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-24T11:17:52.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:51 smithi139 ceph-mon[25719]: pgmap v131: 161 pgs: 4 peering, 157 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-24T11:17:53.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:17:53 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=error ts=2024-03-24T11:17:53.510Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://172.21.15.139:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.139 because it doesn't contain any IP SANs; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://172.21.15.62:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.62 because it doesn't contain any IP SANs" 2024-03-24T11:17:53.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:17:53 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:17:53.512Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.62:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.62 because it doesn't contain any IP SANs" 2024-03-24T11:17:53.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:17:53 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:17:53.513Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.139:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.139 because it doesn't contain any IP SANs" 2024-03-24T11:17:55.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:54 smithi062 ceph-mon[24799]: pgmap v132: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:17:55.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:54 smithi062 ceph-mon[29666]: pgmap v132: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:17:55.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:54 smithi139 ceph-mon[25719]: pgmap v132: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:17:56.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:55 smithi062 ceph-mon[24799]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:17:56.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:55 smithi062 ceph-mon[29666]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:17:56.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:55 smithi139 ceph-mon[25719]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:17:57.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:56 smithi062 ceph-mon[24799]: pgmap v133: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:17:57.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:56 smithi062 ceph-mon[29666]: pgmap v133: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:17:57.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:56 smithi139 ceph-mon[25719]: pgmap v133: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:17:58.977 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:17:58 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.62 - - [24/Mar/2024:11:17:58] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:17:58.977 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:17:58 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:17:58] "GET /metrics HTTP/1.1" 200 214889 "" "Prometheus/2.33.4" 2024-03-24T11:17:58.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:58 smithi062 ceph-mon[24799]: pgmap v134: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-24T11:17:58.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:58 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:58.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:17:58 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:58.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:58 smithi062 ceph-mon[29666]: pgmap v134: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-24T11:17:58.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:58 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:58.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:17:58 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:59.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:58 smithi139 ceph-mon[25719]: pgmap v134: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-24T11:17:59.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:58 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:17:59.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:17:58 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:18:01.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:18:00 smithi139 ceph-mon[25719]: pgmap v135: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 935 B/s rd, 0 op/s 2024-03-24T11:18:01.108 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:18:00 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:18:00] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-03-24T11:18:01.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:18:00 smithi062 ceph-mon[29666]: pgmap v135: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 935 B/s rd, 0 op/s 2024-03-24T11:18:01.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:18:00 smithi062 ceph-mon[24799]: pgmap v135: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 935 B/s rd, 0 op/s 2024-03-24T11:18:03.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:18:02 smithi062 ceph-mon[24799]: pgmap v136: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:03.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:18:02 smithi062 ceph-mon[29666]: pgmap v136: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:03.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:18:02 smithi139 ceph-mon[25719]: pgmap v136: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:03.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:18:03 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=error ts=2024-03-24T11:18:03.511Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://172.21.15.139:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.139 because it doesn't contain any IP SANs; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://172.21.15.62:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.62 because it doesn't contain any IP SANs" 2024-03-24T11:18:03.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:18:03 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:18:03.514Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.62:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.62 because it doesn't contain any IP SANs" 2024-03-24T11:18:03.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:18:03 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:18:03.514Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.139:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.139 because it doesn't contain any IP SANs" 2024-03-24T11:18:05.160 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:18:04 smithi139 ceph-mon[25719]: pgmap v137: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:18:05.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:18:04 smithi062 ceph-mon[24799]: pgmap v137: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:18:05.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:18:04 smithi062 ceph-mon[29666]: pgmap v137: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:18:05.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:18:05 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.139 - - [24/Mar/2024:11:18:05] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:18:06.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:18:05 smithi062 ceph-mon[24799]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:18:06.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:18:05 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:18:06.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:18:05 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:18:06.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:18:05 smithi062 ceph-mon[29666]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:18:06.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:18:05 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:18:06.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:18:05 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:18:06.304 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:18:05 smithi139 ceph-mon[25719]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:18:06.304 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:18:05 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:18:06.304 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:18:05 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:18:07.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:18:06 smithi062 ceph-mon[24799]: pgmap v138: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:07.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:18:06 smithi062 ceph-mon[29666]: pgmap v138: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:07.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:18:06 smithi139 ceph-mon[25719]: pgmap v138: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:09.048 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:18:08 smithi139 ceph-mon[25719]: pgmap v139: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:09.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:18:08 smithi062 ceph-mon[24799]: pgmap v139: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:09.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:18:08 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:18:08] "GET /metrics HTTP/1.1" 200 214861 "" "Prometheus/2.33.4" 2024-03-24T11:18:09.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:18:08 smithi062 ceph-mon[29666]: pgmap v139: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:11.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:18:10 smithi139 ceph-mon[25719]: pgmap v140: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:18:11.108 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:18:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:18:10] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-03-24T11:18:11.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:18:10 smithi062 ceph-mon[24799]: pgmap v140: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:18:11.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:18:10 smithi062 ceph-mon[29666]: pgmap v140: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:18:13.226 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:18:12 smithi062 ceph-mon[29666]: pgmap v141: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:13.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:18:12 smithi062 ceph-mon[24799]: pgmap v141: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:13.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:18:12 smithi139 ceph-mon[25719]: pgmap v141: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:13.976 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:18:13 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=error ts=2024-03-24T11:18:13.513Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://172.21.15.62:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.62 because it doesn't contain any IP SANs; ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://172.21.15.139:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.139 because it doesn't contain any IP SANs" 2024-03-24T11:18:13.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:18:13 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:18:13.516Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.62:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.62 because it doesn't contain any IP SANs" 2024-03-24T11:18:13.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:18:13 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:18:13.516Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.139:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.139 because it doesn't contain any IP SANs" 2024-03-24T11:18:15.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:18:14 smithi062 ceph-mon[24799]: pgmap v142: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:18:15.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:18:14 smithi062 ceph-mon[29666]: pgmap v142: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:18:15.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:18:14 smithi139 ceph-mon[25719]: pgmap v142: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:18:16.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:18:15 smithi062 ceph-mon[24799]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:18:16.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:18:15 smithi062 ceph-mon[29666]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:18:16.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:18:15 smithi139 ceph-mon[25719]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:18:17.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:18:16 smithi062 ceph-mon[24799]: pgmap v143: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:17.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:18:16 smithi062 ceph-mon[29666]: pgmap v143: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:17.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:18:16 smithi139 ceph-mon[25719]: pgmap v143: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:19.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:18:18 smithi062 ceph-mon[29666]: pgmap v144: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:19.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:18:18 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:18:19.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:18:18 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:18:19.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:18:18 smithi062 ceph-mon[24799]: pgmap v144: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:19.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:18:18 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:18:19.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:18:18 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:18:19.228 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:18:18 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.62 - - [24/Mar/2024:11:18:18] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:18:19.228 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:18:18 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:18:18] "GET /metrics HTTP/1.1" 200 214853 "" "Prometheus/2.33.4" 2024-03-24T11:18:19.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:18:18 smithi139 ceph-mon[25719]: pgmap v144: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:19.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:18:18 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:18:19.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:18:18 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:18:21.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:18:20 smithi139 ceph-mon[25719]: pgmap v145: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:18:21.108 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:18:20 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:18:20] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-03-24T11:18:21.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:18:20 smithi062 ceph-mon[24799]: pgmap v145: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:18:21.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:18:20 smithi062 ceph-mon[29666]: pgmap v145: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:18:22.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:18:21 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:18:22.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:18:21 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:18:22.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:18:21 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:18:22.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:18:21 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:18:22.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:18:21 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:18:22.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:18:21 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:18:22.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:18:21 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:18:22.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:18:21 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:18:22.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:18:21 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:18:23.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:18:22 smithi062 ceph-mon[24799]: pgmap v146: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:23.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:18:22 smithi062 ceph-mon[29666]: pgmap v146: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:23.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:18:22 smithi139 ceph-mon[25719]: pgmap v146: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:23.976 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:18:23 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=error ts=2024-03-24T11:18:23.513Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://172.21.15.139:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.139 because it doesn't contain any IP SANs; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://172.21.15.62:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.62 because it doesn't contain any IP SANs" 2024-03-24T11:18:23.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:18:23 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:18:23.515Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.62:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.62 because it doesn't contain any IP SANs" 2024-03-24T11:18:23.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:18:23 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:18:23.515Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.139:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.139 because it doesn't contain any IP SANs" 2024-03-24T11:18:25.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:18:24 smithi062 ceph-mon[29666]: pgmap v147: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:18:25.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:18:24 smithi062 ceph-mon[24799]: pgmap v147: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:18:25.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:18:25 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.139 - - [24/Mar/2024:11:18:25] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:18:25.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:18:24 smithi139 ceph-mon[25719]: pgmap v147: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:18:26.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:18:25 smithi062 ceph-mon[24799]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:18:26.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:18:25 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:18:26.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:18:25 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:18:26.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:18:25 smithi062 ceph-mon[29666]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:18:26.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:18:25 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:18:26.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:18:25 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:18:26.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:18:25 smithi139 ceph-mon[25719]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:18:26.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:18:25 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:18:26.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:18:25 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:18:27.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:18:26 smithi062 ceph-mon[24799]: pgmap v148: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:27.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:18:26 smithi062 ceph-mon[29666]: pgmap v148: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:27.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:18:26 smithi139 ceph-mon[25719]: pgmap v148: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:29.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:18:28 smithi062 ceph-mon[24799]: pgmap v149: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:29.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:18:28 smithi062 ceph-mon[29666]: pgmap v149: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:29.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:18:28 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:18:28] "GET /metrics HTTP/1.1" 200 214853 "" "Prometheus/2.33.4" 2024-03-24T11:18:29.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:18:28 smithi139 ceph-mon[25719]: pgmap v149: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:31.106 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:18:30 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:18:30] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-03-24T11:18:31.106 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:18:30 smithi139 ceph-mon[25719]: pgmap v150: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:18:31.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:18:30 smithi062 ceph-mon[24799]: pgmap v150: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:18:31.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:18:30 smithi062 ceph-mon[29666]: pgmap v150: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:18:32.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:18:31 smithi139 ceph-mon[25719]: pgmap v151: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:32.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:18:31 smithi062 ceph-mon[24799]: pgmap v151: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:32.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:18:32 smithi062 ceph-mon[29666]: pgmap v151: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:33.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:18:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=error ts=2024-03-24T11:18:33.514Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://172.21.15.62:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.62 because it doesn't contain any IP SANs; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://172.21.15.139:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.139 because it doesn't contain any IP SANs" 2024-03-24T11:18:33.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:18:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:18:33.516Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.62:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.62 because it doesn't contain any IP SANs" 2024-03-24T11:18:33.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:18:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:18:33.516Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.139:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.139 because it doesn't contain any IP SANs" 2024-03-24T11:18:35.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:18:34 smithi062 ceph-mon[24799]: pgmap v152: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:18:35.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:18:34 smithi062 ceph-mon[29666]: pgmap v152: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:18:35.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:18:34 smithi139 ceph-mon[25719]: pgmap v152: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:18:36.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:18:35 smithi062 ceph-mon[29666]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:18:36.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:18:35 smithi062 ceph-mon[24799]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:18:36.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:18:35 smithi139 ceph-mon[25719]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:18:37.103 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:18:36 smithi062 ceph-mon[24799]: pgmap v153: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:37.104 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:18:36 smithi062 ceph-mon[29666]: pgmap v153: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:37.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:18:36 smithi139 ceph-mon[25719]: pgmap v153: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:39.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:18:38 smithi062 ceph-mon[24799]: pgmap v154: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:39.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:18:38 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:18:38] "GET /metrics HTTP/1.1" 200 214847 "" "Prometheus/2.33.4" 2024-03-24T11:18:39.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:18:38 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.62 - - [24/Mar/2024:11:18:38] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:18:39.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:18:38 smithi062 ceph-mon[29666]: pgmap v154: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:39.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:18:38 smithi139 ceph-mon[25719]: pgmap v154: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:40.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:18:39 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:18:40.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:18:39 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:18:40.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:18:39 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:18:40.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:18:39 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:18:40.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:18:39 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:18:40.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:18:39 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:18:41.108 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:18:40 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:18:40] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-03-24T11:18:41.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:18:40 smithi139 ceph-mon[25719]: pgmap v155: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:18:41.226 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:18:40 smithi062 ceph-mon[29666]: pgmap v155: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:18:41.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:18:40 smithi062 ceph-mon[24799]: pgmap v155: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:18:43.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:18:42 smithi062 ceph-mon[24799]: pgmap v156: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:43.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:18:42 smithi062 ceph-mon[29666]: pgmap v156: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:43.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:18:42 smithi139 ceph-mon[25719]: pgmap v156: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:43.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:18:43 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=error ts=2024-03-24T11:18:43.515Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://172.21.15.62:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.62 because it doesn't contain any IP SANs; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://172.21.15.139:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.139 because it doesn't contain any IP SANs" 2024-03-24T11:18:43.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:18:43 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:18:43.517Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.62:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.62 because it doesn't contain any IP SANs" 2024-03-24T11:18:43.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:18:43 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:18:43.517Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.139:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.139 because it doesn't contain any IP SANs" 2024-03-24T11:18:45.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:18:44 smithi062 ceph-mon[24799]: pgmap v157: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:18:45.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:18:44 smithi062 ceph-mon[29666]: pgmap v157: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:18:45.266 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:18:44 smithi139 ceph-mon[25719]: pgmap v157: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:18:45.726 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:18:45 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.139 - - [24/Mar/2024:11:18:45] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:18:46.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:18:45 smithi062 ceph-mon[24799]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:18:46.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:18:45 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:18:46.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:18:45 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:18:46.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:18:45 smithi062 ceph-mon[29666]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:18:46.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:18:45 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:18:46.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:18:45 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:18:46.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:18:45 smithi139 ceph-mon[25719]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:18:46.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:18:45 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:18:46.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:18:45 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:18:47.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:18:46 smithi062 ceph-mon[24799]: pgmap v158: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:47.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:18:46 smithi062 ceph-mon[29666]: pgmap v158: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:47.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:18:46 smithi139 ceph-mon[25719]: pgmap v158: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:49.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:18:48 smithi062 ceph-mon[24799]: pgmap v159: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:49.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:18:48 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-24T11:18:49.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:18:48 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-24T11:18:49.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:18:48 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:18:48] "GET /metrics HTTP/1.1" 200 214863 "" "Prometheus/2.33.4" 2024-03-24T11:18:49.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:18:48 smithi062 ceph-mon[29666]: pgmap v159: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:49.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:18:48 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-24T11:18:49.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:18:48 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-24T11:18:49.236 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:18:48 smithi139 ceph-mon[25719]: pgmap v159: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:49.236 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:18:48 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-24T11:18:49.237 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:18:48 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-24T11:18:50.858 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:18:50 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:18:50] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-03-24T11:18:51.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:18:50 smithi062 ceph-mon[24799]: pgmap v160: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:18:51.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:18:50 smithi062 ceph-mon[29666]: pgmap v160: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:18:51.334 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:18:50 smithi139 ceph-mon[25719]: pgmap v160: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:18:53.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:18:52 smithi062 ceph-mon[24799]: pgmap v161: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:53.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:18:52 smithi062 ceph-mon[29666]: pgmap v161: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:53.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:18:52 smithi139 ceph-mon[25719]: pgmap v161: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:53.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:18:53 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=error ts=2024-03-24T11:18:53.515Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://172.21.15.139:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.139 because it doesn't contain any IP SANs; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://172.21.15.62:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.62 because it doesn't contain any IP SANs" 2024-03-24T11:18:53.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:18:53 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:18:53.517Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.62:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.62 because it doesn't contain any IP SANs" 2024-03-24T11:18:53.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:18:53 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:18:53.517Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.139:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.139 because it doesn't contain any IP SANs" 2024-03-24T11:18:55.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:18:54 smithi062 ceph-mon[29666]: pgmap v162: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:18:55.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:18:54 smithi062 ceph-mon[24799]: pgmap v162: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:18:55.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:18:54 smithi139 ceph-mon[25719]: pgmap v162: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:18:56.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:18:55 smithi062 ceph-mon[29666]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:18:56.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:18:55 smithi062 ceph-mon[24799]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:18:56.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:18:55 smithi139 ceph-mon[25719]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:18:57.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:18:56 smithi062 ceph-mon[24799]: pgmap v163: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:57.172 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:18:56 smithi062 ceph-mon[29666]: pgmap v163: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:57.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:18:56 smithi139 ceph-mon[25719]: pgmap v163: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:58.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:18:58 smithi062 ceph-mon[24799]: pgmap v164: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:58.998 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:18:58 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:18:58] "GET /metrics HTTP/1.1" 200 214863 "" "Prometheus/2.33.4" 2024-03-24T11:18:58.998 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:18:58 smithi062 ceph-mon[29666]: pgmap v164: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:18:59.325 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:18:59 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.62 - - [24/Mar/2024:11:18:59] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:18:59.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:18:58 smithi139 ceph-mon[25719]: pgmap v164: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:00.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:00 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:19:00.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:00 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:19:00.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:00 smithi139 ceph-mon[25719]: pgmap v165: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:19:00.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:00 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:19:00.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:00 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:19:00.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:00 smithi062 ceph-mon[24799]: pgmap v165: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:19:00.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:00 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:19:00.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:00 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:19:00.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:00 smithi062 ceph-mon[29666]: pgmap v165: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:19:01.107 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:19:00 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:19:00] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-03-24T11:19:01.574 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-03-24T11:19:03.214 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:02 smithi062 ceph-mon[24799]: pgmap v166: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:03.214 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:02 smithi062 ceph-mon[29666]: pgmap v166: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:03.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:02 smithi139 ceph-mon[25719]: pgmap v166: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:03.809 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:19:03 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=error ts=2024-03-24T11:19:03.516Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://172.21.15.62:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.62 because it doesn't contain any IP SANs; ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://172.21.15.139:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.139 because it doesn't contain any IP SANs" 2024-03-24T11:19:03.809 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:19:03 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:19:03.519Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.62:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.62 because it doesn't contain any IP SANs" 2024-03-24T11:19:03.809 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:19:03 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:19:03.519Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.139:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.139 because it doesn't contain any IP SANs" 2024-03-24T11:19:04.118 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:03 smithi062 ceph-mon[24799]: pgmap v167: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:19:04.119 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:04 smithi062 ceph-mon[29666]: pgmap v167: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:19:04.141 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-03-24T11:19:04.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:04 smithi139 ceph-mon[25719]: pgmap v167: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:19:05.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:05 smithi062 ceph-mon[24799]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:19:05.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:05 smithi062 ceph-mon[29666]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:19:05.259 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:05 smithi139 ceph-mon[25719]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:19:05.695 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:19:05 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.139 - - [24/Mar/2024:11:19:05] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:19:06.726 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:06 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:19:06.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:06 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:19:06.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:06 smithi062 ceph-mon[24799]: pgmap v168: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:06.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:06 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:19:06.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:06 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:19:06.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:06 smithi062 ceph-mon[29666]: pgmap v168: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:06.770 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph config set global log_to_journald false --force' 2024-03-24T11:19:06.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:06 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:19:06.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:06 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:19:06.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:06 smithi139 ceph-mon[25719]: pgmap v168: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:09.182 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:08 smithi062 ceph-mon[24799]: pgmap v169: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:09.182 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:19:08 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:19:08] "GET /metrics HTTP/1.1" 200 214865 "" "Prometheus/2.33.4" 2024-03-24T11:19:09.183 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:08 smithi062 ceph-mon[29666]: pgmap v169: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:09.205 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-03-24T11:19:09.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:08 smithi139 ceph-mon[25719]: pgmap v169: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:11.108 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:19:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:19:10] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-03-24T11:19:11.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:10 smithi139 ceph-mon[25719]: pgmap v170: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:19:11.207 INFO:teuthology.orchestra.run.smithi062.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:19:11.208 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:10 smithi062 ceph-mon[24799]: pgmap v170: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:19:11.208 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:10 smithi062 ceph-mon[29666]: pgmap v170: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:19:11.835 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-24T11:19:11.846 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi062.front.sepia.ceph.com 2024-03-24T11:19:11.846 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-03-24T11:19:12.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:12 smithi062 ceph-mon[29666]: from='client.24967 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:19:12.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:12 smithi062 ceph-mon[29666]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:19:12.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:12 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:19:12.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:12 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:19:12.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:12 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:19:12.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:12 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:19:12.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:12 smithi062 ceph-mon[29666]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:19:12.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:12 smithi062 ceph-mon[29666]: pgmap v171: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:12.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:12 smithi062 ceph-mon[24799]: from='client.24967 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:19:12.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:12 smithi062 ceph-mon[24799]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:19:12.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:12 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:19:12.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:12 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:19:12.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:12 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:19:12.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:12 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:19:12.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:12 smithi062 ceph-mon[24799]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:19:12.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:12 smithi062 ceph-mon[24799]: pgmap v171: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:12.532 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:12 smithi139 ceph-mon[25719]: from='client.24967 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:19:12.532 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:12 smithi139 ceph-mon[25719]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:19:12.532 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:12 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:19:12.532 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:12 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:19:12.533 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:12 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:19:12.533 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:12 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:19:12.533 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:12 smithi139 ceph-mon[25719]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:19:12.533 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:12 smithi139 ceph-mon[25719]: pgmap v171: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:13.724 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:19:13 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=error ts=2024-03-24T11:19:13.517Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://172.21.15.139:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.139 because it doesn't contain any IP SANs; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://172.21.15.62:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.62 because it doesn't contain any IP SANs" 2024-03-24T11:19:13.725 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:19:13 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:19:13.519Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.139:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.139 because it doesn't contain any IP SANs" 2024-03-24T11:19:13.725 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:19:13 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:19:13.520Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.62:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.62 because it doesn't contain any IP SANs" 2024-03-24T11:19:14.257 INFO:teuthology.orchestra.run.smithi062.stdout:true 2024-03-24T11:19:15.186 INFO:teuthology.orchestra.run.smithi062.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T11:19:15.186 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi062 smithi062 running 16s ago 8m - - 2024-03-24T11:19:15.186 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi139 smithi139 running 9s ago 8m - - 2024-03-24T11:19:15.186 INFO:teuthology.orchestra.run.smithi062.stdout:alertmanager.a smithi062 *:9093,9094 running (2m) 16s ago 3m 28.3M - ba2b418f427c 168b140d6416 2024-03-24T11:19:15.186 INFO:teuthology.orchestra.run.smithi062.stdout:grafana.a smithi139 *:3000 running (3m) 9s ago 3m 55.9M - 8.3.5 dad864ee21e9 a0bc435a222c 2024-03-24T11:19:15.186 INFO:teuthology.orchestra.run.smithi062.stdout:iscsi.foo.smithi062.dggsvk smithi062 running (2m) 16s ago 2m 50.5M - 3.5 e1d6a67b021e dc36997f65c5 2024-03-24T11:19:15.186 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.x smithi139 *:8443 running (7m) 9s ago 7m 413M - 17.2.0 e1d6a67b021e 9bbcf96d02d6 2024-03-24T11:19:15.187 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.y smithi062 *:9283 running (9m) 16s ago 9m 466M - 17.2.0 e1d6a67b021e 28a8eee27817 2024-03-24T11:19:15.187 INFO:teuthology.orchestra.run.smithi062.stdout:mon.a smithi062 running (9m) 16s ago 9m 49.1M 2048M 17.2.0 e1d6a67b021e 712f01aa9180 2024-03-24T11:19:15.187 INFO:teuthology.orchestra.run.smithi062.stdout:mon.b smithi139 running (8m) 9s ago 8m 43.5M 2048M 17.2.0 e1d6a67b021e 8945cdb4e6aa 2024-03-24T11:19:15.187 INFO:teuthology.orchestra.run.smithi062.stdout:mon.c smithi062 running (8m) 16s ago 8m 39.8M 2048M 17.2.0 e1d6a67b021e 39110a16ab62 2024-03-24T11:19:15.187 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.a smithi062 *:9100 running (4m) 16s ago 4m 16.6M - 1dbe0e931976 bb795f9b0512 2024-03-24T11:19:15.187 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.b smithi139 *:9100 running (4m) 9s ago 4m 16.5M - 1dbe0e931976 8921c8c780f7 2024-03-24T11:19:15.187 INFO:teuthology.orchestra.run.smithi062.stdout:osd.0 smithi062 running (7m) 16s ago 7m 49.9M 4096M 17.2.0 e1d6a67b021e 8040b30d7170 2024-03-24T11:19:15.187 INFO:teuthology.orchestra.run.smithi062.stdout:osd.1 smithi062 running (7m) 16s ago 7m 51.8M 4096M 17.2.0 e1d6a67b021e e62d22e5dc29 2024-03-24T11:19:15.187 INFO:teuthology.orchestra.run.smithi062.stdout:osd.2 smithi062 running (6m) 16s ago 6m 45.8M 4096M 17.2.0 e1d6a67b021e 714f550103a7 2024-03-24T11:19:15.187 INFO:teuthology.orchestra.run.smithi062.stdout:osd.3 smithi062 running (6m) 16s ago 6m 49.2M 4096M 17.2.0 e1d6a67b021e 1d2868bd5195 2024-03-24T11:19:15.187 INFO:teuthology.orchestra.run.smithi062.stdout:osd.4 smithi139 running (6m) 9s ago 6m 53.9M 4096M 17.2.0 e1d6a67b021e 7568bfa9e74d 2024-03-24T11:19:15.187 INFO:teuthology.orchestra.run.smithi062.stdout:osd.5 smithi139 running (5m) 9s ago 5m 47.2M 4096M 17.2.0 e1d6a67b021e dd0abd599a14 2024-03-24T11:19:15.187 INFO:teuthology.orchestra.run.smithi062.stdout:osd.6 smithi139 running (5m) 9s ago 5m 47.7M 4096M 17.2.0 e1d6a67b021e a4b1289aca69 2024-03-24T11:19:15.188 INFO:teuthology.orchestra.run.smithi062.stdout:osd.7 smithi139 running (4m) 9s ago 4m 49.5M 4096M 17.2.0 e1d6a67b021e bc7fadbc1384 2024-03-24T11:19:15.188 INFO:teuthology.orchestra.run.smithi062.stdout:prometheus.a smithi139 *:9095 running (2m) 9s ago 4m 63.4M - 514e6a882f6e 3ac3b6a49a6e 2024-03-24T11:19:15.188 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi062.sacpds smithi062 *:8000 running (2m) 16s ago 2m 69.6M - 17.2.0 e1d6a67b021e 89f4ac308e98 2024-03-24T11:19:15.188 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi139.yljesh smithi139 *:8000 running (2m) 9s ago 2m 68.8M - 17.2.0 e1d6a67b021e e189bdc12cae 2024-03-24T11:19:15.188 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi062.xvpxnk smithi062 *:80 running (2m) 16s ago 2m 69.4M - 17.2.0 e1d6a67b021e d1aa74adab44 2024-03-24T11:19:15.188 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi139.pbtrmp smithi139 *:80 running (2m) 9s ago 2m 72.1M - 17.2.0 e1d6a67b021e 7f8e336755d7 2024-03-24T11:19:15.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:14 smithi062 ceph-mon[24799]: pgmap v172: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:19:15.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:14 smithi062 ceph-mon[29666]: pgmap v172: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:19:15.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:14 smithi139 ceph-mon[25719]: pgmap v172: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:19:15.730 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:19:15.730 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": { 2024-03-24T11:19:15.730 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 3 2024-03-24T11:19:15.731 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:19:15.731 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": { 2024-03-24T11:19:15.731 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 2 2024-03-24T11:19:15.731 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:19:15.731 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": { 2024-03-24T11:19:15.731 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 8 2024-03-24T11:19:15.731 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:19:15.731 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": {}, 2024-03-24T11:19:15.731 INFO:teuthology.orchestra.run.smithi062.stdout: "rgw": { 2024-03-24T11:19:15.731 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4 2024-03-24T11:19:15.731 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:19:15.731 INFO:teuthology.orchestra.run.smithi062.stdout: "overall": { 2024-03-24T11:19:15.731 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 17 2024-03-24T11:19:15.731 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-24T11:19:15.731 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:19:16.033 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:15 smithi062 ceph-mon[24799]: from='client.15069 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:19:16.033 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:15 smithi062 ceph-mon[24799]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:19:16.033 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:15 smithi062 ceph-mon[24799]: from='client.15075 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:19:16.033 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:15 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/4170627838' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:19:16.033 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:15 smithi062 ceph-mon[29666]: from='client.15069 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:19:16.033 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:15 smithi062 ceph-mon[29666]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:19:16.033 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:15 smithi062 ceph-mon[29666]: from='client.15075 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:19:16.033 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:15 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/4170627838' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:19:16.175 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:19:16.175 INFO:teuthology.orchestra.run.smithi062.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69", 2024-03-24T11:19:16.175 INFO:teuthology.orchestra.run.smithi062.stdout: "in_progress": true, 2024-03-24T11:19:16.175 INFO:teuthology.orchestra.run.smithi062.stdout: "services_complete": [], 2024-03-24T11:19:16.175 INFO:teuthology.orchestra.run.smithi062.stdout: "progress": "", 2024-03-24T11:19:16.175 INFO:teuthology.orchestra.run.smithi062.stdout: "message": "Doing first pull of quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 image" 2024-03-24T11:19:16.175 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:19:16.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:15 smithi139 ceph-mon[25719]: from='client.15069 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:19:16.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:15 smithi139 ceph-mon[25719]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:19:16.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:15 smithi139 ceph-mon[25719]: from='client.15075 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:19:16.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:15 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/4170627838' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:19:16.691 INFO:teuthology.orchestra.run.smithi062.stdout:HEALTH_OK 2024-03-24T11:19:16.958 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:16 smithi062 ceph-mon[24799]: from='client.24985 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:19:16.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:16 smithi062 ceph-mon[24799]: pgmap v173: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:16.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:16 smithi062 ceph-mon[24799]: from='client.24997 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:19:16.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:16 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/3954521046' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:19:16.959 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:16 smithi062 ceph-mon[29666]: from='client.24985 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:19:16.959 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:16 smithi062 ceph-mon[29666]: pgmap v173: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:16.959 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:16 smithi062 ceph-mon[29666]: from='client.24997 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:19:16.959 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:16 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/3954521046' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:19:17.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:16 smithi139 ceph-mon[25719]: from='client.24985 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:19:17.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:16 smithi139 ceph-mon[25719]: pgmap v173: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:17.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:16 smithi139 ceph-mon[25719]: from='client.24997 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:19:17.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:16 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/3954521046' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:19:19.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:18 smithi062 ceph-mon[24799]: pgmap v174: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:19.172 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:19:18 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:19:18] "GET /metrics HTTP/1.1" 200 214847 "" "Prometheus/2.33.4" 2024-03-24T11:19:19.173 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:19:19 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.62 - - [24/Mar/2024:11:19:19] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:19:19.173 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:18 smithi062 ceph-mon[29666]: pgmap v174: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:19.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:18 smithi139 ceph-mon[25719]: pgmap v174: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:20.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:20 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:19:20.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:20 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:19:20.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:20 smithi062 ceph-mon[24799]: pgmap v175: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:19:20.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:20 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:19:20.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:20 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:19:20.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:20 smithi062 ceph-mon[29666]: pgmap v175: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:19:20.607 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:20 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:19:20.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:20 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:19:20.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:20 smithi139 ceph-mon[25719]: pgmap v175: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:19:21.108 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:19:20 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:19:20] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-03-24T11:19:23.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:22 smithi062 ceph-mon[24799]: pgmap v176: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:23.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:22 smithi062 ceph-mon[29666]: pgmap v176: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:23.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:22 smithi139 ceph-mon[25719]: pgmap v176: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:23.976 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:19:23 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=error ts=2024-03-24T11:19:23.518Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://172.21.15.62:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.62 because it doesn't contain any IP SANs; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://172.21.15.139:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.139 because it doesn't contain any IP SANs" 2024-03-24T11:19:23.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:19:23 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:19:23.520Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.62:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.62 because it doesn't contain any IP SANs" 2024-03-24T11:19:23.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:19:23 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:19:23.520Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.139:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.139 because it doesn't contain any IP SANs" 2024-03-24T11:19:24.302 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:24 smithi062 ceph-mon[24799]: pgmap v177: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:19:24.303 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:24 smithi062 ceph-mon[29666]: pgmap v177: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:19:24.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:24 smithi139 ceph-mon[25719]: pgmap v177: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:19:25.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:25 smithi139 ceph-mon[25719]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:19:25.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:25 smithi062 ceph-mon[24799]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:19:25.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:25 smithi062 ceph-mon[29666]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:19:25.976 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:19:25 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.139 - - [24/Mar/2024:11:19:25] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:19:26.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:26 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:19:26.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:26 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:19:26.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:26 smithi139 ceph-mon[25719]: pgmap v178: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:26.915 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:26 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:19:26.915 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:26 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:19:26.916 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:26 smithi062 ceph-mon[24799]: pgmap v178: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:26.916 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:26 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:19:26.916 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:26 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:19:26.916 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:26 smithi062 ceph-mon[29666]: pgmap v178: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:29.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:28 smithi062 ceph-mon[24799]: pgmap v179: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:29.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:19:28 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:19:28] "GET /metrics HTTP/1.1" 200 214847 "" "Prometheus/2.33.4" 2024-03-24T11:19:29.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:28 smithi062 ceph-mon[29666]: pgmap v179: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:29.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:28 smithi139 ceph-mon[25719]: pgmap v179: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:31.041 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:30 smithi139 ceph-mon[25719]: pgmap v180: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:19:31.041 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:19:30 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:19:30] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-03-24T11:19:31.226 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:30 smithi062 ceph-mon[29666]: pgmap v180: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:19:31.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:30 smithi062 ceph-mon[24799]: pgmap v180: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:19:33.221 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:32 smithi139 ceph-mon[25719]: pgmap v181: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:33.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:32 smithi062 ceph-mon[24799]: pgmap v181: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:33.226 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:32 smithi062 ceph-mon[29666]: pgmap v181: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:33.976 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:19:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=error ts=2024-03-24T11:19:33.518Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://172.21.15.139:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.139 because it doesn't contain any IP SANs; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://172.21.15.62:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.62 because it doesn't contain any IP SANs" 2024-03-24T11:19:33.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:19:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:19:33.520Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.139:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.139 because it doesn't contain any IP SANs" 2024-03-24T11:19:33.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:19:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:19:33.520Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.62:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.62 because it doesn't contain any IP SANs" 2024-03-24T11:19:35.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:34 smithi062 ceph-mon[24799]: pgmap v182: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:19:35.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:34 smithi062 ceph-mon[29666]: pgmap v182: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:19:35.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:34 smithi139 ceph-mon[25719]: pgmap v182: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:19:36.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:35 smithi062 ceph-mon[24799]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:19:36.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:35 smithi062 ceph-mon[29666]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:19:36.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:35 smithi139 ceph-mon[25719]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:19:37.168 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:36 smithi062 ceph-mon[24799]: pgmap v183: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:37.169 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:36 smithi062 ceph-mon[29666]: pgmap v183: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:37.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:36 smithi139 ceph-mon[25719]: pgmap v183: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:39.195 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:19:38 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:19:38] "GET /metrics HTTP/1.1" 200 214830 "" "Prometheus/2.33.4" 2024-03-24T11:19:39.195 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:38 smithi062 ceph-mon[24799]: pgmap v184: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:39.195 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:38 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:19:39.195 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:38 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:19:39.195 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:38 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:19:39.195 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:38 smithi062 ceph-mon[29666]: pgmap v184: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:39.195 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:38 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:19:39.195 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:38 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:19:39.196 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:38 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:19:39.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:38 smithi139 ceph-mon[25719]: pgmap v184: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:39.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:38 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:19:39.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:38 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:19:39.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:38 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:19:39.456 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:19:39 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.62 - - [24/Mar/2024:11:19:39] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:19:40.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:39 smithi062 ceph-mon[24799]: Upgrade: Target is version 19.0.0-1667-gdb0330b1 (unknown) 2024-03-24T11:19:40.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:39 smithi062 ceph-mon[24799]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, digests ['quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10'] 2024-03-24T11:19:40.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:39 smithi062 ceph-mon[24799]: Upgrade: Need to upgrade myself (mgr.y) 2024-03-24T11:19:40.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:39 smithi062 ceph-mon[24799]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 on smithi139 2024-03-24T11:19:40.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:39 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:19:40.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:39 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:19:40.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:39 smithi062 ceph-mon[29666]: Upgrade: Target is version 19.0.0-1667-gdb0330b1 (unknown) 2024-03-24T11:19:40.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:39 smithi062 ceph-mon[29666]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, digests ['quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10'] 2024-03-24T11:19:40.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:39 smithi062 ceph-mon[29666]: Upgrade: Need to upgrade myself (mgr.y) 2024-03-24T11:19:40.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:39 smithi062 ceph-mon[29666]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 on smithi139 2024-03-24T11:19:40.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:39 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:19:40.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:39 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:19:40.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:39 smithi139 ceph-mon[25719]: Upgrade: Target is version 19.0.0-1667-gdb0330b1 (unknown) 2024-03-24T11:19:40.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:39 smithi139 ceph-mon[25719]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, digests ['quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10'] 2024-03-24T11:19:40.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:39 smithi139 ceph-mon[25719]: Upgrade: Need to upgrade myself (mgr.y) 2024-03-24T11:19:40.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:39 smithi139 ceph-mon[25719]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 on smithi139 2024-03-24T11:19:40.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:39 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:19:40.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:39 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:19:41.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:40 smithi139 ceph-mon[25719]: pgmap v185: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:19:41.108 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:19:40 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:19:40] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-03-24T11:19:41.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:40 smithi062 ceph-mon[24799]: pgmap v185: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:19:41.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:40 smithi062 ceph-mon[29666]: pgmap v185: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:19:43.226 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:42 smithi062 ceph-mon[29666]: pgmap v186: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:43.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:42 smithi062 ceph-mon[24799]: pgmap v186: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:43.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:42 smithi139 ceph-mon[25719]: pgmap v186: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:43.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:19:43 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=error ts=2024-03-24T11:19:43.519Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://172.21.15.62:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.62 because it doesn't contain any IP SANs; ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://172.21.15.139:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.139 because it doesn't contain any IP SANs" 2024-03-24T11:19:43.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:19:43 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:19:43.521Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.62:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.62 because it doesn't contain any IP SANs" 2024-03-24T11:19:43.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:19:43 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:19:43.521Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.139:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.139 because it doesn't contain any IP SANs" 2024-03-24T11:19:45.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:44 smithi062 ceph-mon[24799]: pgmap v187: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:19:45.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:44 smithi062 ceph-mon[29666]: pgmap v187: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:19:45.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:44 smithi139 ceph-mon[25719]: pgmap v187: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:19:45.917 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:19:45 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.139 - - [24/Mar/2024:11:19:45] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:19:46.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:45 smithi062 ceph-mon[24799]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:19:46.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:45 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:19:46.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:45 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:19:46.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:45 smithi062 ceph-mon[29666]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:19:46.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:45 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:19:46.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:45 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:19:46.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:45 smithi139 ceph-mon[25719]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:19:46.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:45 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:19:46.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:45 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:19:47.031 INFO:teuthology.orchestra.run.smithi062.stdout:true 2024-03-24T11:19:47.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:46 smithi062 ceph-mon[24799]: pgmap v188: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:47.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:46 smithi062 ceph-mon[29666]: pgmap v188: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:47.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:46 smithi139 ceph-mon[25719]: pgmap v188: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:47.632 INFO:teuthology.orchestra.run.smithi062.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T11:19:47.632 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi062 smithi062 running 8s ago 9m - - 2024-03-24T11:19:47.632 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi139 smithi139 running 2s ago 9m - - 2024-03-24T11:19:47.632 INFO:teuthology.orchestra.run.smithi062.stdout:alertmanager.a smithi062 *:9093,9094 running (3m) 8s ago 3m 27.9M - ba2b418f427c 168b140d6416 2024-03-24T11:19:47.632 INFO:teuthology.orchestra.run.smithi062.stdout:grafana.a smithi139 *:3000 running (3m) 2s ago 3m 56.0M - 8.3.5 dad864ee21e9 a0bc435a222c 2024-03-24T11:19:47.632 INFO:teuthology.orchestra.run.smithi062.stdout:iscsi.foo.smithi062.dggsvk smithi062 running (2m) 8s ago 2m 50.7M - 3.5 e1d6a67b021e dc36997f65c5 2024-03-24T11:19:47.632 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.x smithi139 *:8443 running (8m) 2s ago 8m 413M - 17.2.0 e1d6a67b021e 9bbcf96d02d6 2024-03-24T11:19:47.632 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.y smithi062 *:9283 running (10m) 8s ago 10m 467M - 17.2.0 e1d6a67b021e 28a8eee27817 2024-03-24T11:19:47.632 INFO:teuthology.orchestra.run.smithi062.stdout:mon.a smithi062 running (10m) 8s ago 10m 49.9M 2048M 17.2.0 e1d6a67b021e 712f01aa9180 2024-03-24T11:19:47.633 INFO:teuthology.orchestra.run.smithi062.stdout:mon.b smithi139 running (8m) 2s ago 8m 43.7M 2048M 17.2.0 e1d6a67b021e 8945cdb4e6aa 2024-03-24T11:19:47.633 INFO:teuthology.orchestra.run.smithi062.stdout:mon.c smithi062 running (8m) 8s ago 8m 40.2M 2048M 17.2.0 e1d6a67b021e 39110a16ab62 2024-03-24T11:19:47.633 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.a smithi062 *:9100 running (4m) 8s ago 4m 16.8M - 1dbe0e931976 bb795f9b0512 2024-03-24T11:19:47.633 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.b smithi139 *:9100 running (4m) 2s ago 4m 17.0M - 1dbe0e931976 8921c8c780f7 2024-03-24T11:19:47.633 INFO:teuthology.orchestra.run.smithi062.stdout:osd.0 smithi062 running (8m) 8s ago 8m 50.2M 4096M 17.2.0 e1d6a67b021e 8040b30d7170 2024-03-24T11:19:47.633 INFO:teuthology.orchestra.run.smithi062.stdout:osd.1 smithi062 running (7m) 8s ago 7m 52.1M 4096M 17.2.0 e1d6a67b021e e62d22e5dc29 2024-03-24T11:19:47.633 INFO:teuthology.orchestra.run.smithi062.stdout:osd.2 smithi062 running (7m) 8s ago 7m 46.1M 4096M 17.2.0 e1d6a67b021e 714f550103a7 2024-03-24T11:19:47.633 INFO:teuthology.orchestra.run.smithi062.stdout:osd.3 smithi062 running (6m) 8s ago 6m 49.7M 4096M 17.2.0 e1d6a67b021e 1d2868bd5195 2024-03-24T11:19:47.633 INFO:teuthology.orchestra.run.smithi062.stdout:osd.4 smithi139 running (6m) 2s ago 6m 54.2M 4096M 17.2.0 e1d6a67b021e 7568bfa9e74d 2024-03-24T11:19:47.633 INFO:teuthology.orchestra.run.smithi062.stdout:osd.5 smithi139 running (6m) 2s ago 6m 47.6M 4096M 17.2.0 e1d6a67b021e dd0abd599a14 2024-03-24T11:19:47.633 INFO:teuthology.orchestra.run.smithi062.stdout:osd.6 smithi139 running (5m) 2s ago 5m 47.9M 4096M 17.2.0 e1d6a67b021e a4b1289aca69 2024-03-24T11:19:47.633 INFO:teuthology.orchestra.run.smithi062.stdout:osd.7 smithi139 running (5m) 2s ago 5m 49.8M 4096M 17.2.0 e1d6a67b021e bc7fadbc1384 2024-03-24T11:19:47.633 INFO:teuthology.orchestra.run.smithi062.stdout:prometheus.a smithi139 *:9095 running (3m) 2s ago 4m 67.7M - 514e6a882f6e 3ac3b6a49a6e 2024-03-24T11:19:47.634 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi062.sacpds smithi062 *:8000 running (3m) 8s ago 3m 69.8M - 17.2.0 e1d6a67b021e 89f4ac308e98 2024-03-24T11:19:47.634 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi139.yljesh smithi139 *:8000 running (2m) 2s ago 2m 69.0M - 17.2.0 e1d6a67b021e e189bdc12cae 2024-03-24T11:19:47.634 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi062.xvpxnk smithi062 *:80 running (2m) 8s ago 2m 69.6M - 17.2.0 e1d6a67b021e d1aa74adab44 2024-03-24T11:19:47.634 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi139.pbtrmp smithi139 *:80 running (2m) 2s ago 2m 72.2M - 17.2.0 e1d6a67b021e 7f8e336755d7 2024-03-24T11:19:47.994 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:19:47.995 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": { 2024-03-24T11:19:47.995 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 3 2024-03-24T11:19:47.995 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:19:47.995 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": { 2024-03-24T11:19:47.995 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 2 2024-03-24T11:19:47.995 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:19:47.995 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": { 2024-03-24T11:19:47.995 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 8 2024-03-24T11:19:47.995 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:19:47.995 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": {}, 2024-03-24T11:19:47.995 INFO:teuthology.orchestra.run.smithi062.stdout: "rgw": { 2024-03-24T11:19:47.995 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4 2024-03-24T11:19:47.996 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:19:47.996 INFO:teuthology.orchestra.run.smithi062.stdout: "overall": { 2024-03-24T11:19:47.996 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 17 2024-03-24T11:19:47.996 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-24T11:19:47.996 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:19:48.299 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:19:48.300 INFO:teuthology.orchestra.run.smithi062.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69", 2024-03-24T11:19:48.300 INFO:teuthology.orchestra.run.smithi062.stdout: "in_progress": true, 2024-03-24T11:19:48.300 INFO:teuthology.orchestra.run.smithi062.stdout: "services_complete": [], 2024-03-24T11:19:48.300 INFO:teuthology.orchestra.run.smithi062.stdout: "progress": "0/23 daemons upgraded", 2024-03-24T11:19:48.300 INFO:teuthology.orchestra.run.smithi062.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 image on host smithi139" 2024-03-24T11:19:48.300 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:19:48.664 INFO:teuthology.orchestra.run.smithi062.stdout:HEALTH_OK 2024-03-24T11:19:48.977 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:19:48 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:19:48] "GET /metrics HTTP/1.1" 200 214822 "" "Prometheus/2.33.4" 2024-03-24T11:19:48.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:48 smithi062 ceph-mon[24799]: from='client.25009 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:19:48.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:48 smithi062 ceph-mon[24799]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:19:48.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:48 smithi062 ceph-mon[24799]: from='client.25021 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:19:48.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:48 smithi062 ceph-mon[24799]: pgmap v189: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:48.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:48 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-24T11:19:48.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:48 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-24T11:19:48.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:48 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/1539256101' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:19:48.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:48 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/1295262144' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:19:48.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:48 smithi062 ceph-mon[29666]: from='client.25009 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:19:48.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:48 smithi062 ceph-mon[29666]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:19:48.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:48 smithi062 ceph-mon[29666]: from='client.25021 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:19:48.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:48 smithi062 ceph-mon[29666]: pgmap v189: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:48.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:48 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-24T11:19:48.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:48 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-24T11:19:48.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:48 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/1539256101' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:19:48.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:48 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/1295262144' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:19:49.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:48 smithi139 ceph-mon[25719]: from='client.25009 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:19:49.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:48 smithi139 ceph-mon[25719]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:19:49.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:48 smithi139 ceph-mon[25719]: from='client.25021 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:19:49.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:48 smithi139 ceph-mon[25719]: pgmap v189: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:49.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:48 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-24T11:19:49.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:48 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-24T11:19:49.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:48 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/1539256101' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:19:49.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:48 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/1295262144' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:19:50.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:49 smithi062 ceph-mon[24799]: from='client.25033 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:19:50.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:49 smithi062 ceph-mon[29666]: from='client.25033 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:19:50.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:49 smithi139 ceph-mon[25719]: from='client.25033 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:19:51.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:50 smithi139 ceph-mon[25719]: pgmap v190: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:19:51.108 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:19:50 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:19:50] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-03-24T11:19:51.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:50 smithi062 ceph-mon[24799]: pgmap v190: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:19:51.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:50 smithi062 ceph-mon[29666]: pgmap v190: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:19:53.191 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:52 smithi139 ceph-mon[25719]: pgmap v191: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:53.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:52 smithi062 ceph-mon[24799]: pgmap v191: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:53.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:52 smithi062 ceph-mon[29666]: pgmap v191: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:53.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:19:53 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=error ts=2024-03-24T11:19:53.519Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=7 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://172.21.15.62:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.62 because it doesn't contain any IP SANs; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://172.21.15.139:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.139 because it doesn't contain any IP SANs" 2024-03-24T11:19:53.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:19:53 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:19:53.521Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.139:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.139 because it doesn't contain any IP SANs" 2024-03-24T11:19:53.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:19:53 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:19:53.521Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.62:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.62 because it doesn't contain any IP SANs" 2024-03-24T11:19:55.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:54 smithi062 ceph-mon[24799]: pgmap v192: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:19:55.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:54 smithi062 ceph-mon[29666]: pgmap v192: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:19:55.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:54 smithi139 ceph-mon[25719]: pgmap v192: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:19:56.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:55 smithi062 ceph-mon[24799]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:19:56.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:55 smithi062 ceph-mon[29666]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:19:56.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:55 smithi139 ceph-mon[25719]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:19:57.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:56 smithi062 ceph-mon[24799]: pgmap v193: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:57.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:56 smithi062 ceph-mon[29666]: pgmap v193: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:57.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:56 smithi139 ceph-mon[25719]: pgmap v193: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:59.150 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:19:58 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:19:58] "GET /metrics HTTP/1.1" 200 214822 "" "Prometheus/2.33.4" 2024-03-24T11:19:59.151 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:19:58 smithi062 ceph-mon[24799]: pgmap v194: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:59.151 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:19:58 smithi062 ceph-mon[29666]: pgmap v194: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:59.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:19:58 smithi139 ceph-mon[25719]: pgmap v194: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:19:59.428 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:19:59 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.62 - - [24/Mar/2024:11:19:59] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:20:00.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:00 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:20:00.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:00 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:20:00.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:00 smithi062 ceph-mon[24799]: pgmap v195: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:20:00.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:00 smithi062 ceph-mon[24799]: overall HEALTH_OK 2024-03-24T11:20:00.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:00 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:20:00.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:00 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:20:00.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:00 smithi062 ceph-mon[29666]: pgmap v195: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:20:00.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:00 smithi062 ceph-mon[29666]: overall HEALTH_OK 2024-03-24T11:20:00.777 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:00 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:20:00.778 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:00 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:20:00.778 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:00 smithi139 ceph-mon[25719]: pgmap v195: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:20:00.778 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:00 smithi139 ceph-mon[25719]: overall HEALTH_OK 2024-03-24T11:20:01.107 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:00 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:20:00] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-03-24T11:20:02.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:02 smithi139 ceph-mon[25719]: pgmap v196: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:20:02.476 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:02 smithi062 ceph-mon[29666]: pgmap v196: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:20:02.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:02 smithi062 ceph-mon[24799]: pgmap v196: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:20:03.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:20:03 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=error ts=2024-03-24T11:20:03.520Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=7 err="ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://172.21.15.139:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.139 because it doesn't contain any IP SANs; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://172.21.15.62:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.62 because it doesn't contain any IP SANs" 2024-03-24T11:20:03.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:20:03 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:20:03.522Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.139:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.139 because it doesn't contain any IP SANs" 2024-03-24T11:20:03.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:20:03 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:20:03.522Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.62:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.62 because it doesn't contain any IP SANs" 2024-03-24T11:20:04.340 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:04 smithi062 ceph-mon[24799]: pgmap v197: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:20:04.340 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:04 smithi062 ceph-mon[29666]: pgmap v197: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:20:04.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:04 smithi139 ceph-mon[25719]: pgmap v197: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:20:05.259 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:05 smithi139 ceph-mon[25719]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:20:05.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:05 smithi062 ceph-mon[24799]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:20:05.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:05 smithi062 ceph-mon[29666]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:20:06.226 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:20:05 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 172.21.15.139 - - [24/Mar/2024:11:20:05] "POST /data HTTP/1.1" 200 46 "" "Python-urllib/3.9" 2024-03-24T11:20:07.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:06 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:20:07.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:06 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:20:07.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:06 smithi139 ceph-mon[25719]: pgmap v198: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:20:07.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:06 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:20:07.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:06 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:20:07.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:06 smithi062 ceph-mon[24799]: pgmap v198: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:20:07.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:06 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:20:07.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:06 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:20:07.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:06 smithi062 ceph-mon[29666]: pgmap v198: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:20:09.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:08 smithi062 ceph-mon[24799]: pgmap v199: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:20:09.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:08 smithi062 ceph-mon[29666]: pgmap v199: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:20:09.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:20:08 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:20:08] "GET /metrics HTTP/1.1" 200 214791 "" "Prometheus/2.33.4" 2024-03-24T11:20:09.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:08 smithi139 ceph-mon[25719]: pgmap v199: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:20:10.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:10 smithi139 ceph-mon[25719]: pgmap v200: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:20:10.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:10 smithi062 ceph-mon[24799]: pgmap v200: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:20:10.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:10 smithi062 ceph-mon[29666]: pgmap v200: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:20:10.858 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[27231]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:20:10] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-03-24T11:20:13.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:12 smithi139 ceph-mon[25719]: Upgrade: Updating mgr.x 2024-03-24T11:20:13.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:12 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:20:13.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:12 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T11:20:13.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:12 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T11:20:13.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:12 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:20:13.075 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:12 smithi139 ceph-mon[25719]: Deploying daemon mgr.x on smithi139 2024-03-24T11:20:13.075 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:12 smithi139 ceph-mon[25719]: pgmap v201: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:20:13.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:12 smithi062 ceph-mon[24799]: Upgrade: Updating mgr.x 2024-03-24T11:20:13.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:12 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:20:13.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:12 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T11:20:13.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:12 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T11:20:13.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:12 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:20:13.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:12 smithi062 ceph-mon[24799]: Deploying daemon mgr.x on smithi139 2024-03-24T11:20:13.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:12 smithi062 ceph-mon[24799]: pgmap v201: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:20:13.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:12 smithi062 ceph-mon[29666]: Upgrade: Updating mgr.x 2024-03-24T11:20:13.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:12 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:20:13.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:12 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T11:20:13.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:12 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T11:20:13.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:12 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:20:13.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:12 smithi062 ceph-mon[29666]: Deploying daemon mgr.x on smithi139 2024-03-24T11:20:13.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:12 smithi062 ceph-mon[29666]: pgmap v201: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:20:13.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:20:13 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=error ts=2024-03-24T11:20:13.521Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=7 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://172.21.15.139:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.139 because it doesn't contain any IP SANs; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://172.21.15.62:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.62 because it doesn't contain any IP SANs" 2024-03-24T11:20:13.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:20:13 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:20:13.523Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.62:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.62 because it doesn't contain any IP SANs" 2024-03-24T11:20:13.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:20:13 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:20:13.523Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.139:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.139 because it doesn't contain any IP SANs" 2024-03-24T11:20:14.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:14 smithi139 ceph-mon[25719]: pgmap v202: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:20:14.108 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:14 smithi139 systemd[1]: Stopping Ceph mgr.x for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:20:14.351 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:14 smithi062 ceph-mon[24799]: pgmap v202: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:20:14.351 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:14 smithi062 ceph-mon[29666]: pgmap v202: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:20:14.858 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:14 smithi139 bash[52160]: Error: no container with name or ID "ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr.x" found: no such container 2024-03-24T11:20:14.858 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:14 smithi139 podman[52173]: 2024-03-24 11:20:14.747862819 +0000 UTC m=+0.126534661 container died 9bbcf96d02d60a5640bccf72a7c875039e7cecc9b10a40d9e823bfb4af0679c2 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x, io.buildah.version=1.19.8, maintainer=Guillaume Abrioux , com.redhat.license_terms=https://centos.org/legal/licensing-policy/, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, vcs-type=git, architecture=x86_64, vendor=Red Hat, Inc., io.openshift.expose-services=, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=-17.2.0, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, GIT_REPO=https://github.com/ceph/ceph-container.git, com.redhat.component=centos-stream-container, build-date=2022-05-03T08:36:31.336870, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, io.openshift.tags=base centos centos-stream, GIT_BRANCH=HEAD, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, name=centos-stream, version=8, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, RELEASE=HEAD, io.k8s.display-name=CentOS Stream 8, distribution-scope=public, release=754, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly.) 2024-03-24T11:20:15.010 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:20:14 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:20:14.737Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=3 err="Post \"https://172.21.15.139:8443/api/prometheus_receiver\": read tcp 172.21.15.62:55598->172.21.15.139:8443: read: connection reset by peer" 2024-03-24T11:20:15.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:15 smithi139 ceph-mon[25719]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:20:15.358 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:15 smithi139 podman[52173]: 2024-03-24 11:20:15.133848962 +0000 UTC m=+0.512520803 container cleanup 9bbcf96d02d60a5640bccf72a7c875039e7cecc9b10a40d9e823bfb4af0679c2 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x, io.openshift.tags=base centos centos-stream, GIT_BRANCH=HEAD, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., RELEASE=HEAD, name=centos-stream, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, build-date=2022-05-03T08:36:31.336870, io.k8s.display-name=CentOS Stream 8, GIT_CLEAN=True, vcs-type=git, io.openshift.expose-services=, ceph=True, architecture=x86_64, vendor=Red Hat, Inc., maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-17.2.0, io.buildah.version=1.19.8, version=8, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, GIT_REPO=https://github.com/ceph/ceph-container.git, com.redhat.component=centos-stream-container, distribution-scope=public, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, release=754, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream) 2024-03-24T11:20:15.358 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:15 smithi139 bash[52173]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x 2024-03-24T11:20:15.358 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:15 smithi139 bash[52200]: Error: no container with name or ID "ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr.x" found: no such container 2024-03-24T11:20:15.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:15 smithi062 ceph-mon[24799]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:20:15.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:15 smithi062 ceph-mon[29666]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:20:15.810 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:15 smithi139 podman[52187]: 2024-03-24 11:20:15.385395701 +0000 UTC m=+0.639988742 container remove 9bbcf96d02d60a5640bccf72a7c875039e7cecc9b10a40d9e823bfb4af0679c2 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x, RELEASE=HEAD, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, GIT_CLEAN=True, distribution-scope=public, io.openshift.tags=base centos centos-stream, io.openshift.expose-services=, GIT_REPO=https://github.com/ceph/ceph-container.git, io.k8s.display-name=CentOS Stream 8, version=8, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., GIT_BRANCH=HEAD, com.redhat.component=centos-stream-container, name=centos-stream, io.buildah.version=1.19.8, architecture=x86_64, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, maintainer=Guillaume Abrioux , GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, vendor=Red Hat, Inc., com.redhat.license_terms=https://centos.org/legal/licensing-policy/, vcs-type=git, build-date=2022-05-03T08:36:31.336870, ceph=True, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, CEPH_POINT_RELEASE=-17.2.0, release=754, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image) 2024-03-24T11:20:15.810 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:15 smithi139 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mgr.x.service: Main process exited, code=exited, status=143/n/a 2024-03-24T11:20:15.810 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:15 smithi139 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mgr.x.service: Failed with result 'exit-code'. 2024-03-24T11:20:15.810 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:15 smithi139 systemd[1]: Stopped Ceph mgr.x for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:20:15.810 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:15 smithi139 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mgr.x.service: Consumed 16.759s CPU time. 2024-03-24T11:20:15.810 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:15 smithi139 systemd[1]: Starting Ceph mgr.x for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:20:15.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:20:15 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:20:15.675Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=4 err="Post \"https://172.21.15.139:8443/api/prometheus_receiver\": dial tcp 172.21.15.139:8443: connect: connection refused" 2024-03-24T11:20:16.060 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:16 smithi139 ceph-mon[25719]: pgmap v203: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:20:16.060 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:15 smithi139 podman[52303]: 2024-03-24 11:20:15.712792944 +0000 UTC m=+0.016916361 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:20:16.061 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:15 smithi139 podman[52303]: 2024-03-24 11:20:15.836605805 +0000 UTC m=+0.140729203 container create ed2c6bf552c087e96f16fd2133789a4845418fa6f0dc149f47a7cc5e0b8e5b50 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x, RELEASE=squid-db0330b, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-03-24T11:20:16.061 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:16 smithi139 podman[52303]: 2024-03-24 11:20:16.057806132 +0000 UTC m=+0.361929547 container init ed2c6bf552c087e96f16fd2133789a4845418fa6f0dc149f47a7cc5e0b8e5b50 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-db0330b, org.label-schema.schema-version=1.0) 2024-03-24T11:20:16.061 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:16 smithi139 podman[52303]: 2024-03-24 11:20:16.060655882 +0000 UTC m=+0.364779286 container start ed2c6bf552c087e96f16fd2133789a4845418fa6f0dc149f47a7cc5e0b8e5b50 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x, RELEASE=squid-db0330b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=) 2024-03-24T11:20:16.301 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:16 smithi062 ceph-mon[24799]: pgmap v203: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:20:16.301 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:16 smithi062 ceph-mon[29666]: pgmap v203: 161 pgs: 161 active+clean; 585 KiB data, 94 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:20:16.341 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:16 smithi139 ceph-mgr[52321]: -- 172.21.15.139:0/2004598249 <== mon.2 v2:172.21.15.139:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55bfe9d421a0 con 0x55bfe9d18800 2024-03-24T11:20:16.342 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:16 smithi139 bash[52303]: ed2c6bf552c087e96f16fd2133789a4845418fa6f0dc149f47a7cc5e0b8e5b50 2024-03-24T11:20:16.342 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:16 smithi139 systemd[1]: Started Ceph mgr.x for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:20:17.108 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:16 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:20:16.628+0000 7fd9eec68200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-24T11:20:17.108 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:16 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:20:16.731+0000 7fd9eec68200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-24T11:20:17.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:17 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:20:17.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:17 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:20:17.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:17 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:20:17.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:17 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:20:17.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:17 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:20:17.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:17 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:20:17.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:17 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:20:17.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:17 smithi062 ceph-mon[24799]: Upgrade: Need to upgrade myself (mgr.y) 2024-03-24T11:20:17.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:17 smithi062 ceph-mon[24799]: Failing over to other MGR 2024-03-24T11:20:17.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:17 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-03-24T11:20:17.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:17 smithi062 ceph-mon[24799]: osdmap e97: 8 total, 8 up, 8 in 2024-03-24T11:20:17.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:17 smithi062 ceph-mon[24799]: Standby manager daemon y started 2024-03-24T11:20:17.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:17 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:20:17.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:17 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:20:17.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:17 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:20:17.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:17 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:20:17.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:17 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:20:17.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:17 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:20:17.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:17 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:20:17.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:17 smithi062 ceph-mon[29666]: Upgrade: Need to upgrade myself (mgr.y) 2024-03-24T11:20:17.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:17 smithi062 ceph-mon[29666]: Failing over to other MGR 2024-03-24T11:20:17.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:17 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-03-24T11:20:17.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:17 smithi062 ceph-mon[29666]: osdmap e97: 8 total, 8 up, 8 in 2024-03-24T11:20:17.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:17 smithi062 ceph-mon[29666]: Standby manager daemon y started 2024-03-24T11:20:17.505 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:17 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:20:17.418+0000 7fd9eec68200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-24T11:20:17.506 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:17 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:20:17.506 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:17 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:20:17.506 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:17 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:20:17.506 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:17 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:20:17.506 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:17 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' 2024-03-24T11:20:17.506 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:17 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:20:17.506 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:17 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:20:17.506 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:17 smithi139 ceph-mon[25719]: Upgrade: Need to upgrade myself (mgr.y) 2024-03-24T11:20:17.506 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:17 smithi139 ceph-mon[25719]: Failing over to other MGR 2024-03-24T11:20:17.506 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:17 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-03-24T11:20:17.506 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:17 smithi139 ceph-mon[25719]: osdmap e97: 8 total, 8 up, 8 in 2024-03-24T11:20:17.506 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:17 smithi139 ceph-mon[25719]: Standby manager daemon y started 2024-03-24T11:20:17.772 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:17 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:20:17.516+0000 7fd9eec68200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-24T11:20:17.772 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:17 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:20:17.669+0000 7fd9eec68200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-24T11:20:17.773 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:17 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:20:17.771+0000 7fd9eec68200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-24T11:20:17.976 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:20:17 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: ignoring --setuser ceph since I am not root 2024-03-24T11:20:17.976 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:20:17 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: ignoring --setgroup ceph since I am not root 2024-03-24T11:20:17.977 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:20:17 smithi062 ceph-mgr[25046]: -- 172.21.15.62:0/3075209659 <== mon.0 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563c8732c340 con 0x563c87304800 2024-03-24T11:20:18.107 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:17 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:20:17.859+0000 7fd9eec68200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-24T11:20:18.300 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:20:18 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:20:18.044+0000 7f1268646000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-24T11:20:18.300 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:20:18 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:20:18.137+0000 7f1268646000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-24T11:20:18.442 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:18 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:20:18.137+0000 7fd9eec68200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-24T11:20:18.442 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:18 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:20:18.252+0000 7fd9eec68200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-24T11:20:18.442 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:18 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:20:18.342+0000 7fd9eec68200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-24T11:20:18.442 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:18 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:20:18.441+0000 7fd9eec68200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-24T11:20:18.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:18 smithi062 ceph-mon[24799]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd='[{"prefix": "mgr fail", "who": "y"}]': finished 2024-03-24T11:20:18.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:18 smithi062 ceph-mon[24799]: mgrmap e22: x(active, starting, since 1.00005s), standbys: y 2024-03-24T11:20:18.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:18 smithi062 ceph-mon[29666]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd='[{"prefix": "mgr fail", "who": "y"}]': finished 2024-03-24T11:20:18.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:18 smithi062 ceph-mon[29666]: mgrmap e22: x(active, starting, since 1.00005s), standbys: y 2024-03-24T11:20:18.857 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:18 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:20:18.623+0000 7fd9eec68200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-24T11:20:18.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:18 smithi139 ceph-mon[25719]: from='mgr.14430 172.21.15.62:0/3792504871' entity='mgr.y' cmd='[{"prefix": "mgr fail", "who": "y"}]': finished 2024-03-24T11:20:18.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:18 smithi139 ceph-mon[25719]: mgrmap e22: x(active, starting, since 1.00005s), standbys: y 2024-03-24T11:20:19.089 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:20:18 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:20:18.750+0000 7f1268646000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-24T11:20:19.089 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:20:18 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:20:18.848+0000 7f1268646000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-24T11:20:19.089 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:20:18 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:20:18.933+0000 7f1268646000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-24T11:20:19.089 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:20:19 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:20:19.088+0000 7f1268646000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-24T11:20:19.347 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:20:19 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:20:19.170+0000 7f1268646000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-24T11:20:19.348 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:20:19 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:20:19.291+0000 7f1268646000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-24T11:20:19.608 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:20:19 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:20:19.377+0000 7f1268646000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-24T11:20:19.608 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:20:19 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:20:19.556+0000 7f1268646000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-24T11:20:19.857 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:19 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:20:19.601+0000 7fd9eec68200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-24T11:20:19.858 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:19 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:20:19.731+0000 7fd9eec68200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-24T11:20:19.976 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:20:19 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:20:19.647+0000 7f1268646000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-24T11:20:20.355 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:20 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:20:20.067+0000 7fd9eec68200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-24T11:20:20.355 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:20 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:20:20.158+0000 7fd9eec68200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-24T11:20:20.355 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:20 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:20:20.354+0000 7fd9eec68200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-24T11:20:20.607 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:20 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:20:20.542+0000 7fd9eec68200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-24T11:20:20.867 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:20:20 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:20:20.725+0000 7f1268646000 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-24T11:20:20.867 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:20:20 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:20:20.572Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=6 err="Post \"https://172.21.15.62:8443//api/prometheus_receiver\": dial tcp 172.21.15.62:8443: connect: connection refused" 2024-03-24T11:20:21.092 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:20 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:20:20.829+0000 7fd9eec68200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-24T11:20:21.162 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:20:20 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:20:20.866+0000 7f1268646000 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-24T11:20:21.162 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:20:21 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:20:21.070+0000 7f1268646000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-24T11:20:21.162 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:20:21 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:20:21.162+0000 7f1268646000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-24T11:20:21.357 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:21 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:20:21.091+0000 7fd9eec68200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-24T11:20:21.358 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:21 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:20:21.329+0000 7fd9eec68200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-24T11:20:21.476 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:20:21 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:20:21.303+0000 7f1268646000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-24T11:20:21.861 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:20:21 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:20:21.632+0000 7f1268646000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-24T11:20:22.124 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:20:21 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:20:21.860+0000 7f1268646000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-24T11:20:22.124 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:20:21 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:20:21.948+0000 7f1268646000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-24T11:20:22.476 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:20:22 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:20:22.123+0000 7f1268646000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-24T11:20:22.608 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:22 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:20:22.135+0000 7fd9eec68200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-24T11:20:23.085 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:22 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:20:22.728+0000 7fd9eec68200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-24T11:20:23.085 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:22 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:20:22.836+0000 7fd9eec68200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-24T11:20:23.358 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:23 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:20:23.084+0000 7fd9eec68200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-24T11:20:23.358 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:23 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: [24/Mar/2024:11:20:23] ENGINE Bus STARTING 2024-03-24T11:20:23.358 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:23 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: CherryPy Checker: 2024-03-24T11:20:23.358 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:23 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: The Application mounted at '' has an empty config. 2024-03-24T11:20:23.358 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:23 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:20:23.358 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:23 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: [24/Mar/2024:11:20:23] ENGINE Serving on http://:::9283 2024-03-24T11:20:23.358 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:23 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: [24/Mar/2024:11:20:23] ENGINE Bus STARTED 2024-03-24T11:20:23.383 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:20:23 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:20:23.073+0000 7f1268646000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-24T11:20:23.638 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:20:23 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:20:23.383+0000 7f1268646000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-24T11:20:23.638 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:20:23 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:20:23.481+0000 7f1268646000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-24T11:20:23.638 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:20:23 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:20:23.637+0000 7f1268646000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-24T11:20:23.638 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:20:23 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=error ts=2024-03-24T11:20:23.521Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=7 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://172.21.15.62:8443//api/prometheus_receiver\": dial tcp 172.21.15.62:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://172.21.15.139:8443/api/prometheus_receiver\": dial tcp 172.21.15.139:8443: connect: connection refused" 2024-03-24T11:20:23.638 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:20:23 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:20:23.522Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.62:8443//api/prometheus_receiver\": dial tcp 172.21.15.62:8443: connect: connection refused" 2024-03-24T11:20:23.638 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:20:23 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:20:23.524Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.139:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.139 because it doesn't contain any IP SANs" 2024-03-24T11:20:23.910 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:20:23 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: [24/Mar/2024:11:20:23] ENGINE Bus STARTING 2024-03-24T11:20:23.910 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:20:23 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: CherryPy Checker: 2024-03-24T11:20:23.910 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:20:23 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: The Application mounted at '' has an empty config. 2024-03-24T11:20:23.910 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:20:23 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: 2024-03-24T11:20:23.910 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:20:23 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: [24/Mar/2024:11:20:23] ENGINE Serving on http://:::9283 2024-03-24T11:20:23.910 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:20:23 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: [24/Mar/2024:11:20:23] ENGINE Bus STARTED 2024-03-24T11:20:24.175 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:20:23 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:20:23.945Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=2 err="Post \"https://172.21.15.139:8443/api/prometheus_receiver\": dial tcp 172.21.15.139:8443: connect: connection refused" 2024-03-24T11:20:24.175 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:23 smithi062 ceph-mon[24799]: Active manager daemon x restarted 2024-03-24T11:20:24.175 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:23 smithi062 ceph-mon[24799]: Activating manager daemon x 2024-03-24T11:20:24.175 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:23 smithi062 ceph-mon[24799]: from='mgr.? 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-24T11:20:24.175 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:23 smithi062 ceph-mon[24799]: osdmap e98: 8 total, 8 up, 8 in 2024-03-24T11:20:24.176 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:23 smithi062 ceph-mon[24799]: from='mgr.? 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T11:20:24.176 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:23 smithi062 ceph-mon[24799]: from='mgr.? 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-24T11:20:24.176 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:23 smithi062 ceph-mon[24799]: from='mgr.? 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T11:20:24.176 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:23 smithi062 ceph-mon[24799]: Standby manager daemon y restarted 2024-03-24T11:20:24.176 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:23 smithi062 ceph-mon[24799]: Standby manager daemon y started 2024-03-24T11:20:24.176 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:23 smithi062 ceph-mon[24799]: from='mgr.? 172.21.15.62:0/1349069298' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-24T11:20:24.176 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:23 smithi062 ceph-mon[24799]: from='mgr.? 172.21.15.62:0/1349069298' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T11:20:24.176 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:23 smithi062 ceph-mon[24799]: from='mgr.? 172.21.15.62:0/1349069298' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-24T11:20:24.176 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:23 smithi062 ceph-mon[24799]: from='mgr.? 172.21.15.62:0/1349069298' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T11:20:24.176 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:23 smithi062 ceph-mon[29666]: Active manager daemon x restarted 2024-03-24T11:20:24.176 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:23 smithi062 ceph-mon[29666]: Activating manager daemon x 2024-03-24T11:20:24.176 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:23 smithi062 ceph-mon[29666]: from='mgr.? 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-24T11:20:24.177 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:23 smithi062 ceph-mon[29666]: osdmap e98: 8 total, 8 up, 8 in 2024-03-24T11:20:24.177 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:23 smithi062 ceph-mon[29666]: from='mgr.? 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T11:20:24.177 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:23 smithi062 ceph-mon[29666]: from='mgr.? 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-24T11:20:24.177 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:23 smithi062 ceph-mon[29666]: from='mgr.? 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T11:20:24.177 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:23 smithi062 ceph-mon[29666]: Standby manager daemon y restarted 2024-03-24T11:20:24.177 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:23 smithi062 ceph-mon[29666]: Standby manager daemon y started 2024-03-24T11:20:24.177 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:23 smithi062 ceph-mon[29666]: from='mgr.? 172.21.15.62:0/1349069298' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-24T11:20:24.177 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:23 smithi062 ceph-mon[29666]: from='mgr.? 172.21.15.62:0/1349069298' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T11:20:24.177 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:23 smithi062 ceph-mon[29666]: from='mgr.? 172.21.15.62:0/1349069298' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-24T11:20:24.177 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:23 smithi062 ceph-mon[29666]: from='mgr.? 172.21.15.62:0/1349069298' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T11:20:24.218 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:23 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: [24/Mar/2024:11:20:23] ENGINE Bus STOPPING 2024-03-24T11:20:24.218 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:23 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: [24/Mar/2024:11:20:23] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-03-24T11:20:24.218 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:23 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: [24/Mar/2024:11:20:23] ENGINE Bus STOPPED 2024-03-24T11:20:24.218 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:24 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: [24/Mar/2024:11:20:24] ENGINE Bus STARTING 2024-03-24T11:20:24.218 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:24 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: [24/Mar/2024:11:20:24] ENGINE Serving on http://:::9283 2024-03-24T11:20:24.218 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:24 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: [24/Mar/2024:11:20:24] ENGINE Bus STARTED 2024-03-24T11:20:24.219 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:23 smithi139 ceph-mon[25719]: Active manager daemon x restarted 2024-03-24T11:20:24.219 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:23 smithi139 ceph-mon[25719]: Activating manager daemon x 2024-03-24T11:20:24.219 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:23 smithi139 ceph-mon[25719]: from='mgr.? 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-24T11:20:24.219 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:23 smithi139 ceph-mon[25719]: osdmap e98: 8 total, 8 up, 8 in 2024-03-24T11:20:24.219 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:23 smithi139 ceph-mon[25719]: from='mgr.? 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T11:20:24.219 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:23 smithi139 ceph-mon[25719]: from='mgr.? 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-24T11:20:24.219 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:23 smithi139 ceph-mon[25719]: from='mgr.? 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T11:20:24.219 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:23 smithi139 ceph-mon[25719]: Standby manager daemon y restarted 2024-03-24T11:20:24.219 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:23 smithi139 ceph-mon[25719]: Standby manager daemon y started 2024-03-24T11:20:24.219 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:23 smithi139 ceph-mon[25719]: from='mgr.? 172.21.15.62:0/1349069298' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-24T11:20:24.219 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:23 smithi139 ceph-mon[25719]: from='mgr.? 172.21.15.62:0/1349069298' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T11:20:24.219 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:23 smithi139 ceph-mon[25719]: from='mgr.? 172.21.15.62:0/1349069298' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-24T11:20:24.219 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:23 smithi139 ceph-mon[25719]: from='mgr.? 172.21.15.62:0/1349069298' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T11:20:24.476 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:20:24.237Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=2 err="Post \"https://172.21.15.62:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.62 because it doesn't contain any IP SANs" 2024-03-24T11:20:24.952 INFO:teuthology.orchestra.run.smithi062.stdout:true 2024-03-24T11:20:25.226 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:20:24.768Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=3 err="Post \"https://172.21.15.139:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.139 because it doesn't contain any IP SANs" 2024-03-24T11:20:25.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[24799]: mgrmap e23: x(active, starting, since 0.821561s), standbys: y 2024-03-24T11:20:25.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T11:20:25.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T11:20:25.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:20:25.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-24T11:20:25.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-24T11:20:25.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T11:20:25.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T11:20:25.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T11:20:25.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T11:20:25.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T11:20:25.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T11:20:25.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T11:20:25.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T11:20:25.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-24T11:20:25.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-24T11:20:25.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-24T11:20:25.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[24799]: Manager daemon x is now available 2024-03-24T11:20:25.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:25.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[24799]: Queued rgw.foo for migration 2024-03-24T11:20:25.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:25.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[24799]: Queued rgw.smpl for migration 2024-03-24T11:20:25.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[24799]: No Migration is needed for rgw spec: {'placement': {'count': 2}, 'service_id': 'foo', 'service_name': 'rgw.foo', 'service_type': 'rgw', 'spec': {'rgw_frontend_port': 8000, 'rgw_realm': 'r', 'rgw_zone': 'z'}} 2024-03-24T11:20:25.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[24799]: No Migration is needed for rgw spec: {'placement': {'count': 2}, 'service_id': 'smpl', 'service_name': 'rgw.smpl', 'service_type': 'rgw'} 2024-03-24T11:20:25.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:25.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:25.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:20:25.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi062", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:20:25.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[29666]: mgrmap e23: x(active, starting, since 0.821561s), standbys: y 2024-03-24T11:20:25.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T11:20:25.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T11:20:25.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:20:25.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-24T11:20:25.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-24T11:20:25.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T11:20:25.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T11:20:25.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T11:20:25.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T11:20:25.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T11:20:25.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T11:20:25.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T11:20:25.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T11:20:25.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-24T11:20:25.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-24T11:20:25.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-24T11:20:25.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[29666]: Manager daemon x is now available 2024-03-24T11:20:25.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:25.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[29666]: Queued rgw.foo for migration 2024-03-24T11:20:25.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:25.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[29666]: Queued rgw.smpl for migration 2024-03-24T11:20:25.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[29666]: No Migration is needed for rgw spec: {'placement': {'count': 2}, 'service_id': 'foo', 'service_name': 'rgw.foo', 'service_type': 'rgw', 'spec': {'rgw_frontend_port': 8000, 'rgw_realm': 'r', 'rgw_zone': 'z'}} 2024-03-24T11:20:25.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[29666]: No Migration is needed for rgw spec: {'placement': {'count': 2}, 'service_id': 'smpl', 'service_name': 'rgw.smpl', 'service_type': 'rgw'} 2024-03-24T11:20:25.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:25.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:25.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:20:25.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi062", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:20:25.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi139", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:20:25.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi062", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:20:25.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi139", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:20:25.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:20:25.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:20:25.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[29666]: Updating smithi062:/etc/ceph/ceph.conf 2024-03-24T11:20:25.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[29666]: Updating smithi139:/etc/ceph/ceph.conf 2024-03-24T11:20:25.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:20:25.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-24T11:20:25.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-24T11:20:25.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-24T11:20:25.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-24T11:20:25.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:25.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:25.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi139", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:20:25.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi062", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:20:25.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi139", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:20:25.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:20:25.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:20:25.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[24799]: Updating smithi062:/etc/ceph/ceph.conf 2024-03-24T11:20:25.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[24799]: Updating smithi139:/etc/ceph/ceph.conf 2024-03-24T11:20:25.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:20:25.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-24T11:20:25.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-24T11:20:25.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-24T11:20:25.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-24T11:20:25.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:25.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:24 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:25.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:24 smithi139 ceph-mon[25719]: mgrmap e23: x(active, starting, since 0.821561s), standbys: y 2024-03-24T11:20:25.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:24 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T11:20:25.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:24 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T11:20:25.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:24 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:20:25.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:24 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-24T11:20:25.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:24 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-24T11:20:25.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:24 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T11:20:25.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:24 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T11:20:25.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:24 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T11:20:25.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:24 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T11:20:25.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:24 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T11:20:25.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:24 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T11:20:25.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:24 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T11:20:25.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:24 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T11:20:25.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:24 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-24T11:20:25.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:24 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-24T11:20:25.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:24 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-24T11:20:25.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:24 smithi139 ceph-mon[25719]: Manager daemon x is now available 2024-03-24T11:20:25.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:24 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:25.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:24 smithi139 ceph-mon[25719]: Queued rgw.foo for migration 2024-03-24T11:20:25.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:24 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:25.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:24 smithi139 ceph-mon[25719]: Queued rgw.smpl for migration 2024-03-24T11:20:25.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:24 smithi139 ceph-mon[25719]: No Migration is needed for rgw spec: {'placement': {'count': 2}, 'service_id': 'foo', 'service_name': 'rgw.foo', 'service_type': 'rgw', 'spec': {'rgw_frontend_port': 8000, 'rgw_realm': 'r', 'rgw_zone': 'z'}} 2024-03-24T11:20:25.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:24 smithi139 ceph-mon[25719]: No Migration is needed for rgw spec: {'placement': {'count': 2}, 'service_id': 'smpl', 'service_name': 'rgw.smpl', 'service_type': 'rgw'} 2024-03-24T11:20:25.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:24 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:25.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:24 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:25.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:24 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:20:25.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:24 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi062", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:20:25.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:24 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi139", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:20:25.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:24 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi062", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:20:25.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:24 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi139", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:20:25.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:24 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:20:25.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:24 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:20:25.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:24 smithi139 ceph-mon[25719]: Updating smithi062:/etc/ceph/ceph.conf 2024-03-24T11:20:25.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:24 smithi139 ceph-mon[25719]: Updating smithi139:/etc/ceph/ceph.conf 2024-03-24T11:20:25.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:24 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:20:25.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:24 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-24T11:20:25.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:24 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-24T11:20:25.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:24 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-24T11:20:25.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:24 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-24T11:20:25.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:24 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:25.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:24 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:25.648 INFO:teuthology.orchestra.run.smithi062.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T11:20:25.648 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi062 smithi062 running 26s ago 9m - - 2024-03-24T11:20:25.648 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi139 smithi139 running 19s ago 9m - - 2024-03-24T11:20:25.648 INFO:teuthology.orchestra.run.smithi062.stdout:alertmanager.a smithi062 *:9093,9094 running (4m) 26s ago 4m 27.7M - ba2b418f427c 168b140d6416 2024-03-24T11:20:25.648 INFO:teuthology.orchestra.run.smithi062.stdout:grafana.a smithi139 *:3000 running (4m) 19s ago 4m 55.6M - 8.3.5 dad864ee21e9 a0bc435a222c 2024-03-24T11:20:25.648 INFO:teuthology.orchestra.run.smithi062.stdout:iscsi.foo.smithi062.dggsvk smithi062 running (3m) 26s ago 3m 50.7M - 3.5 e1d6a67b021e dc36997f65c5 2024-03-24T11:20:25.648 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.x smithi139 *:8443,9283 starting - - - - 2024-03-24T11:20:25.648 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.y smithi062 *:9283 running (10m) 26s ago 10m 467M - 17.2.0 e1d6a67b021e 28a8eee27817 2024-03-24T11:20:25.648 INFO:teuthology.orchestra.run.smithi062.stdout:mon.a smithi062 running (10m) 26s ago 10m 50.2M 2048M 17.2.0 e1d6a67b021e 712f01aa9180 2024-03-24T11:20:25.649 INFO:teuthology.orchestra.run.smithi062.stdout:mon.b smithi139 running (9m) 19s ago 9m 43.8M 2048M 17.2.0 e1d6a67b021e 8945cdb4e6aa 2024-03-24T11:20:25.649 INFO:teuthology.orchestra.run.smithi062.stdout:mon.c smithi062 running (9m) 26s ago 9m 40.4M 2048M 17.2.0 e1d6a67b021e 39110a16ab62 2024-03-24T11:20:25.649 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.a smithi062 *:9100 running (5m) 26s ago 5m 16.2M - 1dbe0e931976 bb795f9b0512 2024-03-24T11:20:25.649 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.b smithi139 *:9100 running (5m) 19s ago 5m 16.7M - 1dbe0e931976 8921c8c780f7 2024-03-24T11:20:25.649 INFO:teuthology.orchestra.run.smithi062.stdout:osd.0 smithi062 running (8m) 26s ago 8m 50.4M 4096M 17.2.0 e1d6a67b021e 8040b30d7170 2024-03-24T11:20:25.649 INFO:teuthology.orchestra.run.smithi062.stdout:osd.1 smithi062 running (8m) 26s ago 8m 52.2M 4096M 17.2.0 e1d6a67b021e e62d22e5dc29 2024-03-24T11:20:25.649 INFO:teuthology.orchestra.run.smithi062.stdout:osd.2 smithi062 running (8m) 26s ago 8m 46.2M 4096M 17.2.0 e1d6a67b021e 714f550103a7 2024-03-24T11:20:25.649 INFO:teuthology.orchestra.run.smithi062.stdout:osd.3 smithi062 running (7m) 26s ago 7m 49.8M 4096M 17.2.0 e1d6a67b021e 1d2868bd5195 2024-03-24T11:20:25.649 INFO:teuthology.orchestra.run.smithi062.stdout:osd.4 smithi139 running (7m) 19s ago 7m 54.5M 4096M 17.2.0 e1d6a67b021e 7568bfa9e74d 2024-03-24T11:20:25.649 INFO:teuthology.orchestra.run.smithi062.stdout:osd.5 smithi139 running (6m) 19s ago 6m 47.7M 4096M 17.2.0 e1d6a67b021e dd0abd599a14 2024-03-24T11:20:25.649 INFO:teuthology.orchestra.run.smithi062.stdout:osd.6 smithi139 running (6m) 19s ago 6m 48.1M 4096M 17.2.0 e1d6a67b021e a4b1289aca69 2024-03-24T11:20:25.649 INFO:teuthology.orchestra.run.smithi062.stdout:osd.7 smithi139 running (6m) 19s ago 6m 50.0M 4096M 17.2.0 e1d6a67b021e bc7fadbc1384 2024-03-24T11:20:25.649 INFO:teuthology.orchestra.run.smithi062.stdout:prometheus.a smithi139 *:9095 running (4m) 19s ago 5m 67.7M - 514e6a882f6e 3ac3b6a49a6e 2024-03-24T11:20:25.649 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi062.sacpds smithi062 *:8000 running (3m) 26s ago 3m 69.9M - 17.2.0 e1d6a67b021e 89f4ac308e98 2024-03-24T11:20:25.649 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi139.yljesh smithi139 *:8000 running (3m) 19s ago 3m 69.1M - 17.2.0 e1d6a67b021e e189bdc12cae 2024-03-24T11:20:25.650 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi062.xvpxnk smithi062 *:80 running (3m) 26s ago 3m 69.7M - 17.2.0 e1d6a67b021e d1aa74adab44 2024-03-24T11:20:25.650 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi139.pbtrmp smithi139 *:80 running (3m) 19s ago 3m 72.3M - 17.2.0 e1d6a67b021e 7f8e336755d7 2024-03-24T11:20:26.065 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:20:26.065 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": { 2024-03-24T11:20:26.065 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 3 2024-03-24T11:20:26.065 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:20:26.065 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": { 2024-03-24T11:20:26.065 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 1, 2024-03-24T11:20:26.065 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 1 2024-03-24T11:20:26.065 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:20:26.066 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": { 2024-03-24T11:20:26.066 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 8 2024-03-24T11:20:26.066 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:20:26.066 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": {}, 2024-03-24T11:20:26.066 INFO:teuthology.orchestra.run.smithi062.stdout: "rgw": { 2024-03-24T11:20:26.066 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4 2024-03-24T11:20:26.066 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:20:26.066 INFO:teuthology.orchestra.run.smithi062.stdout: "overall": { 2024-03-24T11:20:26.066 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 16, 2024-03-24T11:20:26.066 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 1 2024-03-24T11:20:26.066 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-24T11:20:26.066 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:20:26.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:25 smithi062 ceph-mon[24799]: Updating smithi062:/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T11:20:26.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:25 smithi062 ceph-mon[24799]: [24/Mar/2024:11:20:24] ENGINE Bus STARTING 2024-03-24T11:20:26.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:25 smithi062 ceph-mon[24799]: Updating smithi139:/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T11:20:26.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:25 smithi062 ceph-mon[24799]: [24/Mar/2024:11:20:24] ENGINE Serving on https://172.21.15.139:7150 2024-03-24T11:20:26.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:25 smithi062 ceph-mon[24799]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2024-03-24T11:20:26.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:25 smithi062 ceph-mon[24799]: [24/Mar/2024:11:20:24] ENGINE Serving on http://172.21.15.139:8765 2024-03-24T11:20:26.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:25 smithi062 ceph-mon[24799]: [24/Mar/2024:11:20:24] ENGINE Bus STARTED 2024-03-24T11:20:26.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:25 smithi062 ceph-mon[24799]: mgrmap e24: x(active, since 1.82606s), standbys: y 2024-03-24T11:20:26.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:25 smithi062 ceph-mon[24799]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:20:26.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:25 smithi062 ceph-mon[24799]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:20:26.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:25 smithi062 ceph-mon[24799]: pgmap v3: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:20:26.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:25 smithi062 ceph-mon[24799]: Updating smithi139:/etc/ceph/ceph.client.admin.keyring 2024-03-24T11:20:26.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:25 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:26.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:25 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:26.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:25 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:26.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:25 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:26.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:25 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:26.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:25 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:26.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:25 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:26.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:25 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:26.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:25 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:26.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:25 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:26.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:25 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi062", "caps": []}]: dispatch 2024-03-24T11:20:26.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:25 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi062", "caps": []}]: dispatch 2024-03-24T11:20:26.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:25 smithi062 ceph-mon[29666]: Updating smithi062:/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T11:20:26.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:25 smithi062 ceph-mon[29666]: [24/Mar/2024:11:20:24] ENGINE Bus STARTING 2024-03-24T11:20:26.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:25 smithi062 ceph-mon[29666]: Updating smithi139:/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T11:20:26.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:25 smithi062 ceph-mon[29666]: [24/Mar/2024:11:20:24] ENGINE Serving on https://172.21.15.139:7150 2024-03-24T11:20:26.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:25 smithi062 ceph-mon[29666]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2024-03-24T11:20:26.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:25 smithi062 ceph-mon[29666]: [24/Mar/2024:11:20:24] ENGINE Serving on http://172.21.15.139:8765 2024-03-24T11:20:26.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:25 smithi062 ceph-mon[29666]: [24/Mar/2024:11:20:24] ENGINE Bus STARTED 2024-03-24T11:20:26.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:25 smithi062 ceph-mon[29666]: mgrmap e24: x(active, since 1.82606s), standbys: y 2024-03-24T11:20:26.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:25 smithi062 ceph-mon[29666]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:20:26.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:25 smithi062 ceph-mon[29666]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:20:26.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:25 smithi062 ceph-mon[29666]: pgmap v3: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:20:26.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:25 smithi062 ceph-mon[29666]: Updating smithi139:/etc/ceph/ceph.client.admin.keyring 2024-03-24T11:20:26.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:25 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:26.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:25 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:26.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:25 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:26.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:25 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:26.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:25 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:26.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:25 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:26.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:25 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:26.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:25 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:26.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:25 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:26.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:25 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:26.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:25 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi062", "caps": []}]: dispatch 2024-03-24T11:20:26.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:25 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi062", "caps": []}]: dispatch 2024-03-24T11:20:26.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:25 smithi139 ceph-mon[25719]: Updating smithi062:/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T11:20:26.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:25 smithi139 ceph-mon[25719]: [24/Mar/2024:11:20:24] ENGINE Bus STARTING 2024-03-24T11:20:26.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:25 smithi139 ceph-mon[25719]: Updating smithi139:/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T11:20:26.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:25 smithi139 ceph-mon[25719]: [24/Mar/2024:11:20:24] ENGINE Serving on https://172.21.15.139:7150 2024-03-24T11:20:26.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:25 smithi139 ceph-mon[25719]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2024-03-24T11:20:26.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:25 smithi139 ceph-mon[25719]: [24/Mar/2024:11:20:24] ENGINE Serving on http://172.21.15.139:8765 2024-03-24T11:20:26.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:25 smithi139 ceph-mon[25719]: [24/Mar/2024:11:20:24] ENGINE Bus STARTED 2024-03-24T11:20:26.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:25 smithi139 ceph-mon[25719]: mgrmap e24: x(active, since 1.82606s), standbys: y 2024-03-24T11:20:26.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:25 smithi139 ceph-mon[25719]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:20:26.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:25 smithi139 ceph-mon[25719]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:20:26.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:25 smithi139 ceph-mon[25719]: pgmap v3: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:20:26.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:25 smithi139 ceph-mon[25719]: Updating smithi139:/etc/ceph/ceph.client.admin.keyring 2024-03-24T11:20:26.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:25 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:26.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:25 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:26.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:25 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:26.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:25 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:26.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:25 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:26.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:25 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:26.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:25 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:26.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:25 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:26.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:25 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:26.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:25 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:26.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:25 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi062", "caps": []}]: dispatch 2024-03-24T11:20:26.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:25 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi062", "caps": []}]: dispatch 2024-03-24T11:20:26.400 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:20:26.400 INFO:teuthology.orchestra.run.smithi062.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69", 2024-03-24T11:20:26.400 INFO:teuthology.orchestra.run.smithi062.stdout: "in_progress": true, 2024-03-24T11:20:26.400 INFO:teuthology.orchestra.run.smithi062.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T11:20:26.400 INFO:teuthology.orchestra.run.smithi062.stdout: "services_complete": [], 2024-03-24T11:20:26.400 INFO:teuthology.orchestra.run.smithi062.stdout: "progress": "", 2024-03-24T11:20:26.400 INFO:teuthology.orchestra.run.smithi062.stdout: "message": "", 2024-03-24T11:20:26.401 INFO:teuthology.orchestra.run.smithi062.stdout: "is_paused": false 2024-03-24T11:20:26.401 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:20:26.801 INFO:teuthology.orchestra.run.smithi062.stdout:HEALTH_OK 2024-03-24T11:20:27.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:26 smithi139 ceph-mon[25719]: Updating smithi062:/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T11:20:27.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:26 smithi139 ceph-mon[25719]: from='client.25075 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:20:27.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:26 smithi139 ceph-mon[25719]: Updating smithi139:/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T11:20:27.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:26 smithi139 ceph-mon[25719]: from='client.25081 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:20:27.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:26 smithi139 ceph-mon[25719]: pgmap v4: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:20:27.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:26 smithi139 ceph-mon[25719]: mgrmap e25: x(active, since 2s), standbys: y 2024-03-24T11:20:27.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:26 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/3451815203' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:20:27.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:26 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/339163488' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:20:27.226 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:26 smithi062 ceph-mon[29666]: Updating smithi062:/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T11:20:27.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:26 smithi062 ceph-mon[29666]: from='client.25075 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:20:27.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:26 smithi062 ceph-mon[29666]: Updating smithi139:/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T11:20:27.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:26 smithi062 ceph-mon[29666]: from='client.25081 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:20:27.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:26 smithi062 ceph-mon[29666]: pgmap v4: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:20:27.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:26 smithi062 ceph-mon[29666]: mgrmap e25: x(active, since 2s), standbys: y 2024-03-24T11:20:27.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:26 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/3451815203' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:20:27.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:26 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/339163488' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:20:27.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:26 smithi062 ceph-mon[24799]: Updating smithi062:/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T11:20:27.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:26 smithi062 ceph-mon[24799]: from='client.25075 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:20:27.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:26 smithi062 ceph-mon[24799]: Updating smithi139:/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T11:20:27.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:26 smithi062 ceph-mon[24799]: from='client.25081 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:20:27.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:26 smithi062 ceph-mon[24799]: pgmap v4: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:20:27.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:26 smithi062 ceph-mon[24799]: mgrmap e25: x(active, since 2s), standbys: y 2024-03-24T11:20:27.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:26 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/3451815203' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:20:27.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:26 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/339163488' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:20:28.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:27 smithi062 ceph-mon[24799]: from='client.25099 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:20:28.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:27 smithi062 ceph-mon[24799]: Reconfiguring daemon agent.smithi062 on smithi062 2024-03-24T11:20:28.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:27 smithi062 ceph-mon[24799]: Deploying cephadm binary to smithi062 2024-03-24T11:20:28.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:27 smithi062 ceph-mon[29666]: from='client.25099 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:20:28.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:27 smithi062 ceph-mon[29666]: Reconfiguring daemon agent.smithi062 on smithi062 2024-03-24T11:20:28.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:27 smithi062 ceph-mon[29666]: Deploying cephadm binary to smithi062 2024-03-24T11:20:28.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:27 smithi139 ceph-mon[25719]: from='client.25099 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:20:28.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:27 smithi139 ceph-mon[25719]: Reconfiguring daemon agent.smithi062 on smithi062 2024-03-24T11:20:28.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:27 smithi139 ceph-mon[25719]: Deploying cephadm binary to smithi062 2024-03-24T11:20:29.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:28 smithi062 ceph-mon[24799]: pgmap v5: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:20:29.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:28 smithi062 ceph-mon[24799]: mgrmap e26: x(active, since 4s), standbys: y 2024-03-24T11:20:29.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:28 smithi062 ceph-mon[29666]: pgmap v5: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:20:29.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:28 smithi062 ceph-mon[29666]: mgrmap e26: x(active, since 4s), standbys: y 2024-03-24T11:20:29.231 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:20:28 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:20:28] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-03-24T11:20:29.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:28 smithi139 ceph-mon[25719]: pgmap v5: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:20:29.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:28 smithi139 ceph-mon[25719]: mgrmap e26: x(active, since 4s), standbys: y 2024-03-24T11:20:30.898 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:30 smithi062 systemd[1]: Stopping Ceph node-exporter.a for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:20:31.052 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:30 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:20:30] "GET /metrics HTTP/1.1" 200 35351 "" "Prometheus/2.33.4" 2024-03-24T11:20:31.227 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:30 smithi062 bash[63031]: Error: no container with name or ID "ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter.a" found: no such container 2024-03-24T11:20:31.227 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:31 smithi062 podman[63044]: 2024-03-24 11:20:31.034936236 +0000 UTC m=+0.128287608 container died bb795f9b0512317ac3febed1bcf5ad84def4a905653211fb92c7e95b6e55fe07 (image=quay.io/prometheus/node-exporter:v1.3.1, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-03-24T11:20:31.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:31 smithi062 ceph-mon[29666]: pgmap v6: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-24T11:20:31.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:31 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:31.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:31 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:31.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:31 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:31.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:31 smithi062 ceph-mon[29666]: Reconfiguring node-exporter.a (dependencies changed)... 2024-03-24T11:20:31.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:31 smithi062 ceph-mon[29666]: Reconfiguring daemon node-exporter.a on smithi062 2024-03-24T11:20:31.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:31 smithi062 ceph-mon[24799]: pgmap v6: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-24T11:20:31.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:31 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:31.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:31 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:31.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:31 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:31.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:31 smithi062 ceph-mon[24799]: Reconfiguring node-exporter.a (dependencies changed)... 2024-03-24T11:20:31.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:31 smithi062 ceph-mon[24799]: Reconfiguring daemon node-exporter.a on smithi062 2024-03-24T11:20:31.312 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:31 smithi139 ceph-mon[25719]: pgmap v6: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-24T11:20:31.312 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:31 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:31.312 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:31 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:31.312 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:31 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:31.312 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:31 smithi139 ceph-mon[25719]: Reconfiguring node-exporter.a (dependencies changed)... 2024-03-24T11:20:31.312 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:31 smithi139 ceph-mon[25719]: Reconfiguring daemon node-exporter.a on smithi062 2024-03-24T11:20:31.884 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:31 smithi062 podman[63044]: 2024-03-24 11:20:31.666438964 +0000 UTC m=+0.759790377 container cleanup bb795f9b0512317ac3febed1bcf5ad84def4a905653211fb92c7e95b6e55fe07 (image=quay.io/prometheus/node-exporter:v1.3.1, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-03-24T11:20:31.884 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:31 smithi062 bash[63044]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a 2024-03-24T11:20:32.226 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:31 smithi062 podman[63059]: 2024-03-24 11:20:31.884255928 +0000 UTC m=+0.853673070 container remove bb795f9b0512317ac3febed1bcf5ad84def4a905653211fb92c7e95b6e55fe07 (image=quay.io/prometheus/node-exporter:v1.3.1, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-03-24T11:20:32.227 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:31 smithi062 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-03-24T11:20:32.722 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:32 smithi062 bash[63071]: Error: no container with name or ID "ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter.a" found: no such container 2024-03-24T11:20:32.723 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:32 smithi062 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@node-exporter.a.service: Failed with result 'exit-code'. 2024-03-24T11:20:32.723 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:32 smithi062 systemd[1]: Stopped Ceph node-exporter.a for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:20:32.723 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:32 smithi062 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@node-exporter.a.service: Consumed 2.879s CPU time. 2024-03-24T11:20:32.723 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:32 smithi062 systemd[1]: Starting Ceph node-exporter.a for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:20:33.332 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-mon[24799]: pgmap v7: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-03-24T11:20:33.333 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-mon[29666]: pgmap v7: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-03-24T11:20:33.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:33 smithi139 ceph-mon[25719]: pgmap v7: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-03-24T11:20:33.611 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=error ts=2024-03-24T11:20:33.522Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=7 err="ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://172.21.15.139:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.139 because it doesn't contain any IP SANs; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://172.21.15.62:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.62 because it doesn't contain any IP SANs" 2024-03-24T11:20:33.611 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:20:33.524Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.62:8443//api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.62 because it doesn't contain any IP SANs" 2024-03-24T11:20:33.612 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=warn ts=2024-03-24T11:20:33.524Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://172.21.15.139:8443/api/prometheus_receiver\": x509: cannot validate certificate for 172.21.15.139 because it doesn't contain any IP SANs" 2024-03-24T11:20:33.612 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 podman[63218]: 2024-03-24 11:20:33.235367405 +0000 UTC m=+0.022200859 image pull 1dbe0e931976487e20e5cfb272087e08a9779c88fd5e9617ed7042dd9751ec26 quay.io/prometheus/node-exporter:v1.3.1 2024-03-24T11:20:33.612 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 podman[63218]: 2024-03-24 11:20:33.333874532 +0000 UTC m=+0.120707974 container create 79d5a02a999fc15cd0f03e2d8e2a652d6c9b8027e1077f90df4608a1ce5ec9d8 (image=quay.io/prometheus/node-exporter:v1.3.1, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-03-24T11:20:33.612 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 podman[63218]: 2024-03-24 11:20:33.544304822 +0000 UTC m=+0.331138280 container init 79d5a02a999fc15cd0f03e2d8e2a652d6c9b8027e1077f90df4608a1ce5ec9d8 (image=quay.io/prometheus/node-exporter:v1.3.1, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-03-24T11:20:33.612 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 podman[63218]: 2024-03-24 11:20:33.546981952 +0000 UTC m=+0.333815400 container start 79d5a02a999fc15cd0f03e2d8e2a652d6c9b8027e1077f90df4608a1ce5ec9d8 (image=quay.io/prometheus/node-exporter:v1.3.1, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-03-24T11:20:33.612 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[63253]: ts=2024-03-24T11:20:33.549Z caller=node_exporter.go:182 level=info msg="Starting node_exporter" version="(version=1.3.1, branch=HEAD, revision=a2321e7b940ddcff26873612bccdf7cd4c42b6b6)" 2024-03-24T11:20:33.612 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[63253]: ts=2024-03-24T11:20:33.550Z caller=node_exporter.go:183 level=info msg="Build context" build_context="(go=go1.17.3, user=root@243aafa5525c, date=20211205-11:09:49)" 2024-03-24T11:20:33.613 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[63253]: ts=2024-03-24T11:20:33.550Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+)($|/) 2024-03-24T11:20:33.613 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[63253]: ts=2024-03-24T11:20:33.550Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-03-24T11:20:33.613 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[63253]: ts=2024-03-24T11:20:33.551Z caller=node_exporter.go:108 level=info msg="Enabled collectors" 2024-03-24T11:20:33.613 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[63253]: ts=2024-03-24T11:20:33.551Z caller=node_exporter.go:115 level=info collector=arp 2024-03-24T11:20:33.613 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[63253]: ts=2024-03-24T11:20:33.551Z caller=node_exporter.go:115 level=info collector=bcache 2024-03-24T11:20:33.613 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[63253]: ts=2024-03-24T11:20:33.551Z caller=node_exporter.go:115 level=info collector=bonding 2024-03-24T11:20:33.613 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[63253]: ts=2024-03-24T11:20:33.551Z caller=node_exporter.go:115 level=info collector=btrfs 2024-03-24T11:20:33.613 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[63253]: ts=2024-03-24T11:20:33.551Z caller=node_exporter.go:115 level=info collector=conntrack 2024-03-24T11:20:33.613 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[63253]: ts=2024-03-24T11:20:33.551Z caller=node_exporter.go:115 level=info collector=cpu 2024-03-24T11:20:33.613 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[63253]: ts=2024-03-24T11:20:33.551Z caller=node_exporter.go:115 level=info collector=cpufreq 2024-03-24T11:20:33.613 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[63253]: ts=2024-03-24T11:20:33.551Z caller=node_exporter.go:115 level=info collector=diskstats 2024-03-24T11:20:33.613 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[63253]: ts=2024-03-24T11:20:33.551Z caller=node_exporter.go:115 level=info collector=dmi 2024-03-24T11:20:33.613 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[63253]: ts=2024-03-24T11:20:33.551Z caller=node_exporter.go:115 level=info collector=edac 2024-03-24T11:20:33.613 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[63253]: ts=2024-03-24T11:20:33.551Z caller=node_exporter.go:115 level=info collector=entropy 2024-03-24T11:20:33.613 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[63253]: ts=2024-03-24T11:20:33.551Z caller=node_exporter.go:115 level=info collector=fibrechannel 2024-03-24T11:20:33.614 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[63253]: ts=2024-03-24T11:20:33.551Z caller=node_exporter.go:115 level=info collector=filefd 2024-03-24T11:20:33.614 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[63253]: ts=2024-03-24T11:20:33.551Z caller=node_exporter.go:115 level=info collector=filesystem 2024-03-24T11:20:33.614 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[63253]: ts=2024-03-24T11:20:33.551Z caller=node_exporter.go:115 level=info collector=hwmon 2024-03-24T11:20:33.614 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[63253]: ts=2024-03-24T11:20:33.551Z caller=node_exporter.go:115 level=info collector=infiniband 2024-03-24T11:20:33.614 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[63253]: ts=2024-03-24T11:20:33.551Z caller=node_exporter.go:115 level=info collector=ipvs 2024-03-24T11:20:33.614 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[63253]: ts=2024-03-24T11:20:33.551Z caller=node_exporter.go:115 level=info collector=loadavg 2024-03-24T11:20:33.614 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[63253]: ts=2024-03-24T11:20:33.551Z caller=node_exporter.go:115 level=info collector=mdadm 2024-03-24T11:20:33.614 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[63253]: ts=2024-03-24T11:20:33.551Z caller=node_exporter.go:115 level=info collector=meminfo 2024-03-24T11:20:33.614 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[63253]: ts=2024-03-24T11:20:33.551Z caller=node_exporter.go:115 level=info collector=netclass 2024-03-24T11:20:33.614 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[63253]: ts=2024-03-24T11:20:33.551Z caller=node_exporter.go:115 level=info collector=netdev 2024-03-24T11:20:33.614 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[63253]: ts=2024-03-24T11:20:33.551Z caller=node_exporter.go:115 level=info collector=netstat 2024-03-24T11:20:33.614 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[63253]: ts=2024-03-24T11:20:33.551Z caller=node_exporter.go:115 level=info collector=nfs 2024-03-24T11:20:33.614 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[63253]: ts=2024-03-24T11:20:33.551Z caller=node_exporter.go:115 level=info collector=nfsd 2024-03-24T11:20:33.614 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[63253]: ts=2024-03-24T11:20:33.551Z caller=node_exporter.go:115 level=info collector=nvme 2024-03-24T11:20:33.615 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[63253]: ts=2024-03-24T11:20:33.551Z caller=node_exporter.go:115 level=info collector=os 2024-03-24T11:20:33.615 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[63253]: ts=2024-03-24T11:20:33.551Z caller=node_exporter.go:115 level=info collector=powersupplyclass 2024-03-24T11:20:33.615 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[63253]: ts=2024-03-24T11:20:33.551Z caller=node_exporter.go:115 level=info collector=pressure 2024-03-24T11:20:33.615 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[63253]: ts=2024-03-24T11:20:33.551Z caller=node_exporter.go:115 level=info collector=rapl 2024-03-24T11:20:33.615 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[63253]: ts=2024-03-24T11:20:33.551Z caller=node_exporter.go:115 level=info collector=schedstat 2024-03-24T11:20:33.615 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[63253]: ts=2024-03-24T11:20:33.551Z caller=node_exporter.go:115 level=info collector=sockstat 2024-03-24T11:20:33.615 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[63253]: ts=2024-03-24T11:20:33.551Z caller=node_exporter.go:115 level=info collector=softnet 2024-03-24T11:20:33.615 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[63253]: ts=2024-03-24T11:20:33.551Z caller=node_exporter.go:115 level=info collector=stat 2024-03-24T11:20:33.615 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[63253]: ts=2024-03-24T11:20:33.551Z caller=node_exporter.go:115 level=info collector=tapestats 2024-03-24T11:20:33.615 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[63253]: ts=2024-03-24T11:20:33.551Z caller=node_exporter.go:115 level=info collector=textfile 2024-03-24T11:20:33.615 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[63253]: ts=2024-03-24T11:20:33.551Z caller=node_exporter.go:115 level=info collector=thermal_zone 2024-03-24T11:20:33.616 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[63253]: ts=2024-03-24T11:20:33.551Z caller=node_exporter.go:115 level=info collector=time 2024-03-24T11:20:33.616 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[63253]: ts=2024-03-24T11:20:33.551Z caller=node_exporter.go:115 level=info collector=udp_queues 2024-03-24T11:20:33.616 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[63253]: ts=2024-03-24T11:20:33.551Z caller=node_exporter.go:115 level=info collector=uname 2024-03-24T11:20:33.616 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[63253]: ts=2024-03-24T11:20:33.551Z caller=node_exporter.go:115 level=info collector=vmstat 2024-03-24T11:20:33.616 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[63253]: ts=2024-03-24T11:20:33.551Z caller=node_exporter.go:115 level=info collector=xfs 2024-03-24T11:20:33.616 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[63253]: ts=2024-03-24T11:20:33.551Z caller=node_exporter.go:115 level=info collector=zfs 2024-03-24T11:20:33.616 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[63253]: ts=2024-03-24T11:20:33.551Z caller=node_exporter.go:199 level=info msg="Listening on" address=:9100 2024-03-24T11:20:33.616 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a[63253]: ts=2024-03-24T11:20:33.551Z caller=tls_config.go:195 level=info msg="TLS is disabled." http2=false 2024-03-24T11:20:33.616 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 bash[63218]: 79d5a02a999fc15cd0f03e2d8e2a652d6c9b8027e1077f90df4608a1ce5ec9d8 2024-03-24T11:20:33.897 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:20:33 smithi062 systemd[1]: Started Ceph node-exporter.a for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:20:34.942 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:34 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:34.942 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:34 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:34.942 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:34 smithi062 ceph-mon[24799]: Reconfiguring alertmanager.a (dependencies changed)... 2024-03-24T11:20:34.942 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:34 smithi062 ceph-mon[24799]: Reconfiguring daemon alertmanager.a on smithi062 2024-03-24T11:20:34.942 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:34 smithi062 ceph-mon[24799]: pgmap v8: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2024-03-24T11:20:34.942 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:34 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:34.943 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:34 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:34.943 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:34 smithi062 ceph-mon[29666]: Reconfiguring alertmanager.a (dependencies changed)... 2024-03-24T11:20:34.943 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:34 smithi062 ceph-mon[29666]: Reconfiguring daemon alertmanager.a on smithi062 2024-03-24T11:20:34.943 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:34 smithi062 ceph-mon[29666]: pgmap v8: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2024-03-24T11:20:35.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:34 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:35.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:34 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:35.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:34 smithi139 ceph-mon[25719]: Reconfiguring alertmanager.a (dependencies changed)... 2024-03-24T11:20:35.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:34 smithi139 ceph-mon[25719]: Reconfiguring daemon alertmanager.a on smithi062 2024-03-24T11:20:35.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:34 smithi139 ceph-mon[25719]: pgmap v8: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2024-03-24T11:20:35.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:35 smithi062 ceph-mon[24799]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:20:35.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:35 smithi062 ceph-mon[29666]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:20:35.980 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:35 smithi139 ceph-mon[25719]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:20:36.976 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:36 smithi062 ceph-mon[24799]: pgmap v9: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-24T11:20:36.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:36 smithi062 ceph-mon[29666]: pgmap v9: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-24T11:20:37.058 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:36 smithi139 ceph-mon[25719]: pgmap v9: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-24T11:20:39.192 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:38 smithi139 ceph-mon[25719]: pgmap v10: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-24T11:20:39.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:38 smithi062 ceph-mon[24799]: pgmap v10: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-24T11:20:39.226 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:38 smithi062 ceph-mon[29666]: pgmap v10: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-24T11:20:39.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:20:38 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:20:38] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-03-24T11:20:40.062 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:39 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:20:40.178 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:39 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:20:40.179 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:39 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:20:41.108 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:40 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:20:40] "GET /metrics HTTP/1.1" 200 38154 "" "Prometheus/2.33.4" 2024-03-24T11:20:41.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:40 smithi139 ceph-mon[25719]: pgmap v11: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-24T11:20:41.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:40 smithi062 ceph-mon[24799]: pgmap v11: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-24T11:20:41.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:40 smithi062 ceph-mon[29666]: pgmap v11: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-24T11:20:42.608 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:20:42 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[46110]: ts=2024-03-24T11:20:42.313Z caller=manager.go:609 level=warn component="rule manager" group=pools msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on(pool_id) group_right()\n ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: |\n Pool '{{ $labels.name }}' will be full in less than 5 days assuming the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed it's capacity\n" err="found duplicate series for the match group {pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"172.21.15.62:9283\", job=\"ceph\", pool_id=\"1\"}, {instance=\"172.21.15.139:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-24T11:20:42.829 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:20:42 smithi062 systemd[1]: Stopping Ceph alertmanager.a for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:20:42.830 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:20:42 smithi062 bash[63949]: Error: no container with name or ID "ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager.a" found: no such container 2024-03-24T11:20:43.164 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:42 smithi062 ceph-mon[24799]: pgmap v12: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-24T11:20:43.164 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:20:42 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[50979]: level=info ts=2024-03-24T11:20:42.829Z caller=main.go:557 msg="Received SIGTERM, exiting gracefully..." 2024-03-24T11:20:43.164 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:20:42 smithi062 podman[63960]: 2024-03-24 11:20:42.880771604 +0000 UTC m=+0.170107459 container died 168b140d64163b79e7ff94ca02791ed073bca0d30319bde0244b1cc69a88d845 (image=quay.io/prometheus/alertmanager:v0.23.0, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-24T11:20:43.164 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:42 smithi062 ceph-mon[29666]: pgmap v12: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-24T11:20:43.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:42 smithi139 ceph-mon[25719]: pgmap v12: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-24T11:20:43.432 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:20:43 smithi062 podman[63960]: 2024-03-24 11:20:43.405281689 +0000 UTC m=+0.694617586 container cleanup 168b140d64163b79e7ff94ca02791ed073bca0d30319bde0244b1cc69a88d845 (image=quay.io/prometheus/alertmanager:v0.23.0, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-24T11:20:43.432 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:20:43 smithi062 bash[63960]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a 2024-03-24T11:20:43.727 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:20:43 smithi062 bash[63987]: Error: no container with name or ID "ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager.a" found: no such container 2024-03-24T11:20:43.727 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:20:43 smithi062 podman[63975]: 2024-03-24 11:20:43.631531249 +0000 UTC m=+0.747511080 container remove 168b140d64163b79e7ff94ca02791ed073bca0d30319bde0244b1cc69a88d845 (image=quay.io/prometheus/alertmanager:v0.23.0, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-24T11:20:44.041 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:20:43 smithi062 podman[63975]: 2024-03-24 11:20:43.732331843 +0000 UTC m=+0.848311687 volume remove ad92132d55dcfdd6dcbe63948733f278052b0a0e98593a4217d2266f5ff2047f 2024-03-24T11:20:44.041 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:20:43 smithi062 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@alertmanager.a.service: Deactivated successfully. 2024-03-24T11:20:44.041 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:20:43 smithi062 systemd[1]: Stopped Ceph alertmanager.a for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:20:44.042 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:20:43 smithi062 systemd[1]: Starting Ceph alertmanager.a for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:20:44.042 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:20:43 smithi062 podman[64058]: 2024-03-24 11:20:43.949826364 +0000 UTC m=+0.101722089 volume create ab49cf39fcca8ea02c4e77946c23f3ba20d60a15f62bad37adbca3364df3c102 2024-03-24T11:20:44.042 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:20:43 smithi062 podman[64058]: 2024-03-24 11:20:43.867042539 +0000 UTC m=+0.018938285 image pull ba2b418f427c0636d654de8757e830c80168e76482bcc46bb2138e569d6c91d4 quay.io/prometheus/alertmanager:v0.23.0 2024-03-24T11:20:44.042 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:20:44 smithi062 podman[64058]: 2024-03-24 11:20:44.041909817 +0000 UTC m=+0.193805538 container create 818298d581404f0af2ff188f5c2c67ff02a44a4e5126f4f6bba0475239dbbf76 (image=quay.io/prometheus/alertmanager:v0.23.0, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-24T11:20:44.727 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:20:44 smithi062 podman[64058]: 2024-03-24 11:20:44.319644864 +0000 UTC m=+0.471540599 container init 818298d581404f0af2ff188f5c2c67ff02a44a4e5126f4f6bba0475239dbbf76 (image=quay.io/prometheus/alertmanager:v0.23.0, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-24T11:20:44.727 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:20:44 smithi062 podman[64058]: 2024-03-24 11:20:44.322237968 +0000 UTC m=+0.474133693 container start 818298d581404f0af2ff188f5c2c67ff02a44a4e5126f4f6bba0475239dbbf76 (image=quay.io/prometheus/alertmanager:v0.23.0, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-24T11:20:44.727 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:20:44 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[64072]: level=info ts=2024-03-24T11:20:44.332Z caller=main.go:225 msg="Starting Alertmanager" version="(version=0.23.0, branch=HEAD, revision=61046b17771a57cfd4c4a51be370ab930a4d7d54)" 2024-03-24T11:20:44.727 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:20:44 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[64072]: level=info ts=2024-03-24T11:20:44.332Z caller=main.go:226 build_context="(go=go1.16.7, user=root@e21a959be8d2, date=20210825-10:48:55)" 2024-03-24T11:20:44.727 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:20:44 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[64072]: level=info ts=2024-03-24T11:20:44.333Z caller=cluster.go:184 component=cluster msg="setting advertise address explicitly" addr=172.21.15.62 port=9094 2024-03-24T11:20:44.727 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:20:44 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[64072]: level=info ts=2024-03-24T11:20:44.334Z caller=cluster.go:671 component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-03-24T11:20:44.727 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:20:44 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[64072]: level=info ts=2024-03-24T11:20:44.360Z caller=coordinator.go:113 component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-03-24T11:20:44.727 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:20:44 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[64072]: level=info ts=2024-03-24T11:20:44.360Z caller=coordinator.go:126 component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-03-24T11:20:44.727 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:20:44 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[64072]: level=info ts=2024-03-24T11:20:44.362Z caller=main.go:518 msg=Listening address=:9093 2024-03-24T11:20:44.728 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:20:44 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[64072]: level=info ts=2024-03-24T11:20:44.362Z caller=tls_config.go:191 msg="TLS is disabled." http2=false 2024-03-24T11:20:44.728 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:20:44 smithi062 bash[64058]: 818298d581404f0af2ff188f5c2c67ff02a44a4e5126f4f6bba0475239dbbf76 2024-03-24T11:20:44.728 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:20:44 smithi062 systemd[1]: Started Ceph alertmanager.a for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:20:45.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:44 smithi062 ceph-mon[24799]: pgmap v13: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:20:45.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:44 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:45.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:44 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:45.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:44 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi062.dggsvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-03-24T11:20:45.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:44 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi062.dggsvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-03-24T11:20:45.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:44 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:20:45.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:44 smithi062 ceph-mon[29666]: pgmap v13: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:20:45.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:44 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:45.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:44 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:45.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:44 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi062.dggsvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-03-24T11:20:45.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:44 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi062.dggsvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-03-24T11:20:45.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:44 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:20:45.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:44 smithi139 ceph-mon[25719]: pgmap v13: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:20:45.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:44 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:45.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:44 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:45.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:44 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi062.dggsvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-03-24T11:20:45.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:44 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi062.dggsvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-03-24T11:20:45.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:44 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:20:46.251 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:45 smithi062 ceph-mon[24799]: Reconfiguring iscsi.foo.smithi062.dggsvk (dependencies changed)... 2024-03-24T11:20:46.251 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:45 smithi062 ceph-mon[24799]: Reconfiguring daemon iscsi.foo.smithi062.dggsvk on smithi062 2024-03-24T11:20:46.251 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:45 smithi062 ceph-mon[24799]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:20:46.251 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:45 smithi062 ceph-mon[29666]: Reconfiguring iscsi.foo.smithi062.dggsvk (dependencies changed)... 2024-03-24T11:20:46.251 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:45 smithi062 ceph-mon[29666]: Reconfiguring daemon iscsi.foo.smithi062.dggsvk on smithi062 2024-03-24T11:20:46.251 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:45 smithi062 ceph-mon[29666]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:20:46.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:45 smithi139 ceph-mon[25719]: Reconfiguring iscsi.foo.smithi062.dggsvk (dependencies changed)... 2024-03-24T11:20:46.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:45 smithi139 ceph-mon[25719]: Reconfiguring daemon iscsi.foo.smithi062.dggsvk on smithi062 2024-03-24T11:20:46.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:45 smithi139 ceph-mon[25719]: from='client.24838 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:20:46.541 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:20:46 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[64072]: level=info ts=2024-03-24T11:20:46.334Z caller=cluster.go:696 component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000231235s 2024-03-24T11:20:47.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:46 smithi062 ceph-mon[24799]: pgmap v14: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:20:47.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:46 smithi062 ceph-mon[29666]: pgmap v14: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:20:47.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:46 smithi139 ceph-mon[25719]: pgmap v14: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:20:48.358 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:20:47.923+0000 7fd974920700 -1 log_channel(cephadm) log [ERR] : cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi062.dggsvk ... 2024-03-24T11:20:48.358 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: Creating ceph-iscsi config... 2024-03-24T11:20:48.358 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/iscsi-gateway.cfg 2024-03-24T11:20:48.358 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/tcmu-runner-entrypoint.sh 2024-03-24T11:20:48.358 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: Non-zero exit code 1 from systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk 2024-03-24T11:20:48.358 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: systemctl: stderr Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:20:48.358 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: systemctl: stderr See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:20:48.359 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: Traceback (most recent call last): 2024-03-24T11:20:48.359 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-24T11:20:48.359 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: return _run_code(code, main_globals, None, 2024-03-24T11:20:48.359 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-24T11:20:48.359 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: exec(code, run_globals) 2024-03-24T11:20:48.359 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5520, in 2024-03-24T11:20:48.359 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5508, in main 2024-03-24T11:20:48.359 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3038, in command_deploy_from 2024-03-24T11:20:48.359 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3073, in _common_deploy 2024-03-24T11:20:48.359 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3093, in _deploy_daemon_container 2024-03-24T11:20:48.359 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 1065, in deploy_daemon 2024-03-24T11:20:48.359 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-03-24T11:20:48.360 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: RuntimeError: Failed command: systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk: Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:20:48.360 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:20:48.360 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: Traceback (most recent call last): 2024-03-24T11:20:48.360 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1109, in _check_daemons 2024-03-24T11:20:48.360 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: self.mgr._daemon_action(daemon_spec, action=action) 2024-03-24T11:20:48.360 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/usr/share/ceph/mgr/cephadm/module.py", line 2372, in _daemon_action 2024-03-24T11:20:48.360 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: CephadmServe(self)._create_daemon(daemon_spec, reconfig=(action == 'reconfig'))) 2024-03-24T11:20:48.360 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/usr/share/ceph/mgr/cephadm/module.py", line 739, in wait_async 2024-03-24T11:20:48.360 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: return self.event_loop.get_result(coro, timeout) 2024-03-24T11:20:48.360 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/usr/share/ceph/mgr/cephadm/ssh.py", line 64, in get_result 2024-03-24T11:20:48.360 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: return future.result(timeout) 2024-03-24T11:20:48.360 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/lib64/python3.6/concurrent/futures/_base.py", line 432, in result 2024-03-24T11:20:48.361 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: return self.__get_result() 2024-03-24T11:20:48.361 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/lib64/python3.6/concurrent/futures/_base.py", line 384, in __get_result 2024-03-24T11:20:48.361 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: raise self._exception 2024-03-24T11:20:48.361 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1369, in _create_daemon 2024-03-24T11:20:48.361 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: config_blobs=daemon_spec.final_config, 2024-03-24T11:20:48.361 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1671, in _run_cephadm 2024-03-24T11:20:48.361 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: f'cephadm exited with an error code: {code}, stderr: {err}') 2024-03-24T11:20:48.361 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: orchestrator._interface.OrchestratorError: cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi062.dggsvk ... 2024-03-24T11:20:48.361 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: Creating ceph-iscsi config... 2024-03-24T11:20:48.361 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/iscsi-gateway.cfg 2024-03-24T11:20:48.361 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/tcmu-runner-entrypoint.sh 2024-03-24T11:20:48.361 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: Non-zero exit code 1 from systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk 2024-03-24T11:20:48.361 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: systemctl: stderr Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:20:48.361 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: systemctl: stderr See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:20:48.361 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: Traceback (most recent call last): 2024-03-24T11:20:48.361 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-24T11:20:48.361 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: return _run_code(code, main_globals, None, 2024-03-24T11:20:48.362 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-24T11:20:48.362 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: exec(code, run_globals) 2024-03-24T11:20:48.362 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5520, in 2024-03-24T11:20:48.362 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5508, in main 2024-03-24T11:20:48.362 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3038, in command_deploy_from 2024-03-24T11:20:48.362 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3073, in _common_deploy 2024-03-24T11:20:48.362 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3093, in _deploy_daemon_container 2024-03-24T11:20:48.362 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 1065, in deploy_daemon 2024-03-24T11:20:48.362 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-03-24T11:20:48.362 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: RuntimeError: Failed command: systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk: Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:20:48.362 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:20:49.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:20:48 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:20:48] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-03-24T11:20:49.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:48 smithi062 ceph-mon[24799]: cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi062.dggsvk ... 2024-03-24T11:20:49.227 INFO:journalctl@ceph.mon.a.smithi062.stdout: Creating ceph-iscsi config... 2024-03-24T11:20:49.227 INFO:journalctl@ceph.mon.a.smithi062.stdout: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/iscsi-gateway.cfg 2024-03-24T11:20:49.227 INFO:journalctl@ceph.mon.a.smithi062.stdout: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/tcmu-runner-entrypoint.sh 2024-03-24T11:20:49.227 INFO:journalctl@ceph.mon.a.smithi062.stdout: Non-zero exit code 1 from systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk 2024-03-24T11:20:49.228 INFO:journalctl@ceph.mon.a.smithi062.stdout: systemctl: stderr Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:20:49.228 INFO:journalctl@ceph.mon.a.smithi062.stdout: systemctl: stderr See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:20:49.228 INFO:journalctl@ceph.mon.a.smithi062.stdout: Traceback (most recent call last): 2024-03-24T11:20:49.228 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-24T11:20:49.228 INFO:journalctl@ceph.mon.a.smithi062.stdout: return _run_code(code, main_globals, None, 2024-03-24T11:20:49.228 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-24T11:20:49.228 INFO:journalctl@ceph.mon.a.smithi062.stdout: exec(code, run_globals) 2024-03-24T11:20:49.228 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5520, in 2024-03-24T11:20:49.228 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5508, in main 2024-03-24T11:20:49.228 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3038, in command_deploy_from 2024-03-24T11:20:49.228 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3073, in _common_deploy 2024-03-24T11:20:49.228 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3093, in _deploy_daemon_container 2024-03-24T11:20:49.228 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 1065, in deploy_daemon 2024-03-24T11:20:49.229 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-03-24T11:20:49.229 INFO:journalctl@ceph.mon.a.smithi062.stdout: RuntimeError: Failed command: systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk: Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:20:49.229 INFO:journalctl@ceph.mon.a.smithi062.stdout: See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:20:49.229 INFO:journalctl@ceph.mon.a.smithi062.stdout: Traceback (most recent call last): 2024-03-24T11:20:49.229 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1109, in _check_daemons 2024-03-24T11:20:49.229 INFO:journalctl@ceph.mon.a.smithi062.stdout: self.mgr._daemon_action(daemon_spec, action=action) 2024-03-24T11:20:49.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:48 smithi062 ceph-mon[29666]: cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi062.dggsvk ... 2024-03-24T11:20:49.229 INFO:journalctl@ceph.mon.c.smithi062.stdout: Creating ceph-iscsi config... 2024-03-24T11:20:49.229 INFO:journalctl@ceph.mon.c.smithi062.stdout: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/iscsi-gateway.cfg 2024-03-24T11:20:49.229 INFO:journalctl@ceph.mon.c.smithi062.stdout: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/tcmu-runner-entrypoint.sh 2024-03-24T11:20:49.229 INFO:journalctl@ceph.mon.c.smithi062.stdout: Non-zero exit code 1 from systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk 2024-03-24T11:20:49.230 INFO:journalctl@ceph.mon.c.smithi062.stdout: systemctl: stderr Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:20:49.230 INFO:journalctl@ceph.mon.c.smithi062.stdout: systemctl: stderr See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:20:49.230 INFO:journalctl@ceph.mon.c.smithi062.stdout: Traceback (most recent call last): 2024-03-24T11:20:49.230 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-24T11:20:49.230 INFO:journalctl@ceph.mon.c.smithi062.stdout: return _run_code(code, main_globals, None, 2024-03-24T11:20:49.230 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-24T11:20:49.230 INFO:journalctl@ceph.mon.c.smithi062.stdout: exec(code, run_globals) 2024-03-24T11:20:49.230 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5520, in 2024-03-24T11:20:49.230 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5508, in main 2024-03-24T11:20:49.230 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3038, in command_deploy_from 2024-03-24T11:20:49.230 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3073, in _common_deploy 2024-03-24T11:20:49.230 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3093, in _deploy_daemon_container 2024-03-24T11:20:49.230 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 1065, in deploy_daemon 2024-03-24T11:20:49.230 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-03-24T11:20:49.231 INFO:journalctl@ceph.mon.c.smithi062.stdout: RuntimeError: Failed command: systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk: Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:20:49.231 INFO:journalctl@ceph.mon.c.smithi062.stdout: See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:20:49.231 INFO:journalctl@ceph.mon.c.smithi062.stdout: Traceback (most recent call last): 2024-03-24T11:20:49.231 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1109, in _check_daemons 2024-03-24T11:20:49.231 INFO:journalctl@ceph.mon.c.smithi062.stdout: self.mgr._daemon_action(daemon_spec, action=action) 2024-03-24T11:20:49.231 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/usr/share/ceph/mgr/cephadm/module.py", line 2372, in _daemon_action 2024-03-24T11:20:49.231 INFO:journalctl@ceph.mon.c.smithi062.stdout: CephadmServe(self)._create_daemon(daemon_spec, reconfig=(action == 'reconfig'))) 2024-03-24T11:20:49.231 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/usr/share/ceph/mgr/cephadm/module.py", line 739, in wait_async 2024-03-24T11:20:49.231 INFO:journalctl@ceph.mon.c.smithi062.stdout: return self.event_loop.get_result(coro, timeout) 2024-03-24T11:20:49.231 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/usr/share/ceph/mgr/cephadm/ssh.py", line 64, in get_result 2024-03-24T11:20:49.231 INFO:journalctl@ceph.mon.c.smithi062.stdout: return future.result(timeout) 2024-03-24T11:20:49.231 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/lib64/python3.6/concurrent/futures/_base.py", line 432, in result 2024-03-24T11:20:49.231 INFO:journalctl@ceph.mon.c.smithi062.stdout: return self.__get_result() 2024-03-24T11:20:49.231 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/lib64/python3.6/concurrent/futures/_base.py", line 384, in __get_result 2024-03-24T11:20:49.232 INFO:journalctl@ceph.mon.c.smithi062.stdout: raise self._exception 2024-03-24T11:20:49.232 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1369, in _create_daemon 2024-03-24T11:20:49.232 INFO:journalctl@ceph.mon.c.smithi062.stdout: config_blobs=daemon_spec.final_config, 2024-03-24T11:20:49.232 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1671, in _run_cephadm 2024-03-24T11:20:49.232 INFO:journalctl@ceph.mon.c.smithi062.stdout: f'cephadm exited with an error code: {code}, stderr: {err}') 2024-03-24T11:20:49.232 INFO:journalctl@ceph.mon.c.smithi062.stdout: orchestrator._interface.OrchestratorError: cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi062.dggsvk ... 2024-03-24T11:20:49.232 INFO:journalctl@ceph.mon.c.smithi062.stdout: Creating ceph-iscsi config... 2024-03-24T11:20:49.232 INFO:journalctl@ceph.mon.c.smithi062.stdout: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/iscsi-gateway.cfg 2024-03-24T11:20:49.232 INFO:journalctl@ceph.mon.c.smithi062.stdout: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/tcmu-runner-entrypoint.sh 2024-03-24T11:20:49.232 INFO:journalctl@ceph.mon.c.smithi062.stdout: Non-zero exit code 1 from systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk 2024-03-24T11:20:49.232 INFO:journalctl@ceph.mon.c.smithi062.stdout: systemctl: stderr Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:20:49.232 INFO:journalctl@ceph.mon.c.smithi062.stdout: systemctl: stderr See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:20:49.232 INFO:journalctl@ceph.mon.c.smithi062.stdout: Traceback (most recent call last): 2024-03-24T11:20:49.232 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-24T11:20:49.233 INFO:journalctl@ceph.mon.c.smithi062.stdout: return _run_code(code, main_globals, None, 2024-03-24T11:20:49.233 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-24T11:20:49.233 INFO:journalctl@ceph.mon.c.smithi062.stdout: exec(code, run_globals) 2024-03-24T11:20:49.233 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5520, in 2024-03-24T11:20:49.233 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5508, in main 2024-03-24T11:20:49.233 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3038, in command_deploy_from 2024-03-24T11:20:49.233 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3073, in _common_deploy 2024-03-24T11:20:49.233 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/usr/share/ceph/mgr/cephadm/module.py", line 2372, in _daemon_action 2024-03-24T11:20:49.233 INFO:journalctl@ceph.mon.a.smithi062.stdout: CephadmServe(self)._create_daemon(daemon_spec, reconfig=(action == 'reconfig'))) 2024-03-24T11:20:49.233 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/usr/share/ceph/mgr/cephadm/module.py", line 739, in wait_async 2024-03-24T11:20:49.234 INFO:journalctl@ceph.mon.a.smithi062.stdout: return self.event_loop.get_result(coro, timeout) 2024-03-24T11:20:49.234 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/usr/share/ceph/mgr/cephadm/ssh.py", line 64, in get_result 2024-03-24T11:20:49.234 INFO:journalctl@ceph.mon.a.smithi062.stdout: return future.result(timeout) 2024-03-24T11:20:49.234 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib64/python3.6/concurrent/futures/_base.py", line 432, in result 2024-03-24T11:20:49.234 INFO:journalctl@ceph.mon.a.smithi062.stdout: return self.__get_result() 2024-03-24T11:20:49.234 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib64/python3.6/concurrent/futures/_base.py", line 384, in __get_result 2024-03-24T11:20:49.234 INFO:journalctl@ceph.mon.a.smithi062.stdout: raise self._exception 2024-03-24T11:20:49.234 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1369, in _create_daemon 2024-03-24T11:20:49.234 INFO:journalctl@ceph.mon.a.smithi062.stdout: config_blobs=daemon_spec.final_config, 2024-03-24T11:20:49.234 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1671, in _run_cephadm 2024-03-24T11:20:49.234 INFO:journalctl@ceph.mon.a.smithi062.stdout: f'cephadm exited with an error code: {code}, stderr: {err}') 2024-03-24T11:20:49.234 INFO:journalctl@ceph.mon.a.smithi062.stdout: orchestrator._interface.OrchestratorError: cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi062.dggsvk ... 2024-03-24T11:20:49.234 INFO:journalctl@ceph.mon.a.smithi062.stdout: Creating ceph-iscsi config... 2024-03-24T11:20:49.234 INFO:journalctl@ceph.mon.a.smithi062.stdout: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/iscsi-gateway.cfg 2024-03-24T11:20:49.235 INFO:journalctl@ceph.mon.a.smithi062.stdout: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/tcmu-runner-entrypoint.sh 2024-03-24T11:20:49.235 INFO:journalctl@ceph.mon.a.smithi062.stdout: Non-zero exit code 1 from systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk 2024-03-24T11:20:49.235 INFO:journalctl@ceph.mon.a.smithi062.stdout: systemctl: stderr Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:20:49.235 INFO:journalctl@ceph.mon.a.smithi062.stdout: systemctl: stderr See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:20:49.235 INFO:journalctl@ceph.mon.a.smithi062.stdout: Traceback (most recent call last): 2024-03-24T11:20:49.235 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-24T11:20:49.235 INFO:journalctl@ceph.mon.a.smithi062.stdout: return _run_code(code, main_globals, None, 2024-03-24T11:20:49.235 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-24T11:20:49.235 INFO:journalctl@ceph.mon.a.smithi062.stdout: exec(code, run_globals) 2024-03-24T11:20:49.235 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5520, in 2024-03-24T11:20:49.235 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5508, in main 2024-03-24T11:20:49.235 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3038, in command_deploy_from 2024-03-24T11:20:49.235 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3073, in _common_deploy 2024-03-24T11:20:49.235 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3093, in _deploy_daemon_container 2024-03-24T11:20:49.236 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 1065, in deploy_daemon 2024-03-24T11:20:49.236 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-03-24T11:20:49.236 INFO:journalctl@ceph.mon.a.smithi062.stdout: RuntimeError: Failed command: systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk: Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:20:49.236 INFO:journalctl@ceph.mon.a.smithi062.stdout: See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:20:49.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:48 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi139", "caps": []}]: dispatch 2024-03-24T11:20:49.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:48 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi139", "caps": []}]: dispatch 2024-03-24T11:20:49.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:48 smithi062 ceph-mon[24799]: pgmap v15: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:20:49.236 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3093, in _deploy_daemon_container 2024-03-24T11:20:49.236 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 1065, in deploy_daemon 2024-03-24T11:20:49.236 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-03-24T11:20:49.236 INFO:journalctl@ceph.mon.c.smithi062.stdout: RuntimeError: Failed command: systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk: Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:20:49.236 INFO:journalctl@ceph.mon.c.smithi062.stdout: See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:20:49.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:48 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi139", "caps": []}]: dispatch 2024-03-24T11:20:49.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:48 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi139", "caps": []}]: dispatch 2024-03-24T11:20:49.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:48 smithi062 ceph-mon[29666]: pgmap v15: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:20:49.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:48 smithi139 ceph-mon[25719]: cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi062.dggsvk ... 2024-03-24T11:20:49.358 INFO:journalctl@ceph.mon.b.smithi139.stdout: Creating ceph-iscsi config... 2024-03-24T11:20:49.358 INFO:journalctl@ceph.mon.b.smithi139.stdout: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/iscsi-gateway.cfg 2024-03-24T11:20:49.358 INFO:journalctl@ceph.mon.b.smithi139.stdout: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/tcmu-runner-entrypoint.sh 2024-03-24T11:20:49.358 INFO:journalctl@ceph.mon.b.smithi139.stdout: Non-zero exit code 1 from systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk 2024-03-24T11:20:49.358 INFO:journalctl@ceph.mon.b.smithi139.stdout: systemctl: stderr Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:20:49.358 INFO:journalctl@ceph.mon.b.smithi139.stdout: systemctl: stderr See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:20:49.358 INFO:journalctl@ceph.mon.b.smithi139.stdout: Traceback (most recent call last): 2024-03-24T11:20:49.359 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-24T11:20:49.359 INFO:journalctl@ceph.mon.b.smithi139.stdout: return _run_code(code, main_globals, None, 2024-03-24T11:20:49.359 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-24T11:20:49.359 INFO:journalctl@ceph.mon.b.smithi139.stdout: exec(code, run_globals) 2024-03-24T11:20:49.359 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5520, in 2024-03-24T11:20:49.359 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5508, in main 2024-03-24T11:20:49.359 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3038, in command_deploy_from 2024-03-24T11:20:49.359 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3073, in _common_deploy 2024-03-24T11:20:49.359 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3093, in _deploy_daemon_container 2024-03-24T11:20:49.359 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 1065, in deploy_daemon 2024-03-24T11:20:49.359 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-03-24T11:20:49.359 INFO:journalctl@ceph.mon.b.smithi139.stdout: RuntimeError: Failed command: systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk: Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:20:49.359 INFO:journalctl@ceph.mon.b.smithi139.stdout: See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:20:49.359 INFO:journalctl@ceph.mon.b.smithi139.stdout: Traceback (most recent call last): 2024-03-24T11:20:49.360 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1109, in _check_daemons 2024-03-24T11:20:49.360 INFO:journalctl@ceph.mon.b.smithi139.stdout: self.mgr._daemon_action(daemon_spec, action=action) 2024-03-24T11:20:49.360 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/usr/share/ceph/mgr/cephadm/module.py", line 2372, in _daemon_action 2024-03-24T11:20:49.360 INFO:journalctl@ceph.mon.b.smithi139.stdout: CephadmServe(self)._create_daemon(daemon_spec, reconfig=(action == 'reconfig'))) 2024-03-24T11:20:49.360 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/usr/share/ceph/mgr/cephadm/module.py", line 739, in wait_async 2024-03-24T11:20:49.360 INFO:journalctl@ceph.mon.b.smithi139.stdout: return self.event_loop.get_result(coro, timeout) 2024-03-24T11:20:49.360 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/usr/share/ceph/mgr/cephadm/ssh.py", line 64, in get_result 2024-03-24T11:20:49.360 INFO:journalctl@ceph.mon.b.smithi139.stdout: return future.result(timeout) 2024-03-24T11:20:49.360 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/lib64/python3.6/concurrent/futures/_base.py", line 432, in result 2024-03-24T11:20:49.360 INFO:journalctl@ceph.mon.b.smithi139.stdout: return self.__get_result() 2024-03-24T11:20:49.360 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/lib64/python3.6/concurrent/futures/_base.py", line 384, in __get_result 2024-03-24T11:20:49.360 INFO:journalctl@ceph.mon.b.smithi139.stdout: raise self._exception 2024-03-24T11:20:49.360 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1369, in _create_daemon 2024-03-24T11:20:49.360 INFO:journalctl@ceph.mon.b.smithi139.stdout: config_blobs=daemon_spec.final_config, 2024-03-24T11:20:49.360 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1671, in _run_cephadm 2024-03-24T11:20:49.361 INFO:journalctl@ceph.mon.b.smithi139.stdout: f'cephadm exited with an error code: {code}, stderr: {err}') 2024-03-24T11:20:49.361 INFO:journalctl@ceph.mon.b.smithi139.stdout: orchestrator._interface.OrchestratorError: cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi062.dggsvk ... 2024-03-24T11:20:49.361 INFO:journalctl@ceph.mon.b.smithi139.stdout: Creating ceph-iscsi config... 2024-03-24T11:20:49.361 INFO:journalctl@ceph.mon.b.smithi139.stdout: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/iscsi-gateway.cfg 2024-03-24T11:20:49.361 INFO:journalctl@ceph.mon.b.smithi139.stdout: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/tcmu-runner-entrypoint.sh 2024-03-24T11:20:49.361 INFO:journalctl@ceph.mon.b.smithi139.stdout: Non-zero exit code 1 from systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk 2024-03-24T11:20:49.361 INFO:journalctl@ceph.mon.b.smithi139.stdout: systemctl: stderr Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:20:49.361 INFO:journalctl@ceph.mon.b.smithi139.stdout: systemctl: stderr See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:20:49.361 INFO:journalctl@ceph.mon.b.smithi139.stdout: Traceback (most recent call last): 2024-03-24T11:20:49.361 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-24T11:20:49.361 INFO:journalctl@ceph.mon.b.smithi139.stdout: return _run_code(code, main_globals, None, 2024-03-24T11:20:49.361 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-24T11:20:49.361 INFO:journalctl@ceph.mon.b.smithi139.stdout: exec(code, run_globals) 2024-03-24T11:20:49.361 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5520, in 2024-03-24T11:20:49.362 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5508, in main 2024-03-24T11:20:49.362 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3038, in command_deploy_from 2024-03-24T11:20:49.362 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3073, in _common_deploy 2024-03-24T11:20:49.362 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3093, in _deploy_daemon_container 2024-03-24T11:20:49.362 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 1065, in deploy_daemon 2024-03-24T11:20:49.362 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-03-24T11:20:49.362 INFO:journalctl@ceph.mon.b.smithi139.stdout: RuntimeError: Failed command: systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk: Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:20:49.362 INFO:journalctl@ceph.mon.b.smithi139.stdout: See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:20:49.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:48 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi139", "caps": []}]: dispatch 2024-03-24T11:20:49.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:48 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi139", "caps": []}]: dispatch 2024-03-24T11:20:49.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:48 smithi139 ceph-mon[25719]: pgmap v15: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:20:50.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:49 smithi062 ceph-mon[24799]: Reconfiguring daemon agent.smithi139 on smithi139 2024-03-24T11:20:50.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:49 smithi062 ceph-mon[24799]: Deploying cephadm binary to smithi139 2024-03-24T11:20:50.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:49 smithi062 ceph-mon[29666]: Reconfiguring daemon agent.smithi139 on smithi139 2024-03-24T11:20:50.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:49 smithi062 ceph-mon[29666]: Deploying cephadm binary to smithi139 2024-03-24T11:20:50.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:49 smithi139 ceph-mon[25719]: Reconfiguring daemon agent.smithi139 on smithi139 2024-03-24T11:20:50.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:49 smithi139 ceph-mon[25719]: Deploying cephadm binary to smithi139 2024-03-24T11:20:51.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:50 smithi139 ceph-mon[25719]: pgmap v16: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-24T11:20:51.108 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:20:50 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:20:50] "GET /metrics HTTP/1.1" 200 38154 "" "Prometheus/2.33.4" 2024-03-24T11:20:51.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:50 smithi062 ceph-mon[24799]: pgmap v16: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-24T11:20:51.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:50 smithi062 ceph-mon[29666]: pgmap v16: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-24T11:20:52.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:52 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:52.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:52 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:52.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:52 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:52.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:52 smithi139 ceph-mon[25719]: Reconfiguring prometheus.a (dependencies changed)... 2024-03-24T11:20:52.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:52 smithi139 ceph-mon[25719]: Reconfiguring daemon prometheus.a on smithi139 2024-03-24T11:20:52.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:52 smithi139 ceph-mon[25719]: pgmap v17: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-03-24T11:20:52.608 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:20:52 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[46110]: ts=2024-03-24T11:20:52.314Z caller=manager.go:609 level=warn component="rule manager" group=pools msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on(pool_id) group_right()\n ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: |\n Pool '{{ $labels.name }}' will be full in less than 5 days assuming the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed it's capacity\n" err="found duplicate series for the match group {pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"172.21.15.62:9283\", job=\"ceph\", pool_id=\"1\"}, {instance=\"172.21.15.139:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-24T11:20:52.726 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:52 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:52.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:52 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:52.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:52 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:52.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:52 smithi062 ceph-mon[24799]: Reconfiguring prometheus.a (dependencies changed)... 2024-03-24T11:20:52.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:52 smithi062 ceph-mon[24799]: Reconfiguring daemon prometheus.a on smithi139 2024-03-24T11:20:52.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:52 smithi062 ceph-mon[24799]: pgmap v17: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-03-24T11:20:52.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:52 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:52.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:52 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:52.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:52 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:20:52.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:52 smithi062 ceph-mon[29666]: Reconfiguring prometheus.a (dependencies changed)... 2024-03-24T11:20:52.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:52 smithi062 ceph-mon[29666]: Reconfiguring daemon prometheus.a on smithi139 2024-03-24T11:20:52.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:52 smithi062 ceph-mon[29666]: pgmap v17: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-03-24T11:20:54.727 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:20:54 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[64072]: level=info ts=2024-03-24T11:20:54.337Z caller=cluster.go:688 component=cluster msg="gossip settled; proceeding" elapsed=10.002410398s 2024-03-24T11:20:55.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:54 smithi062 ceph-mon[24799]: pgmap v18: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-03-24T11:20:55.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:54 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:20:55.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:54 smithi062 ceph-mon[29666]: pgmap v18: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-03-24T11:20:55.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:54 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:20:55.293 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:54 smithi139 ceph-mon[25719]: pgmap v18: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-03-24T11:20:55.293 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:54 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:20:57.164 INFO:teuthology.orchestra.run.smithi062.stdout:true 2024-03-24T11:20:57.332 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:56 smithi139 ceph-mon[25719]: pgmap v19: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-03-24T11:20:57.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:56 smithi062 ceph-mon[24799]: pgmap v19: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-03-24T11:20:57.476 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:56 smithi062 ceph-mon[29666]: pgmap v19: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-03-24T11:20:57.822 INFO:teuthology.orchestra.run.smithi062.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T11:20:57.823 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi062 smithi062 starting - - - - 2024-03-24T11:20:57.823 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi139 smithi139 starting - - - - 2024-03-24T11:20:57.823 INFO:teuthology.orchestra.run.smithi062.stdout:alertmanager.a smithi062 *:9093,9094 starting - - - - 2024-03-24T11:20:57.823 INFO:teuthology.orchestra.run.smithi062.stdout:grafana.a smithi139 *:3000 running (4m) 52s ago 4m 55.6M - 8.3.5 dad864ee21e9 a0bc435a222c 2024-03-24T11:20:57.823 INFO:teuthology.orchestra.run.smithi062.stdout:iscsi.foo.smithi062.dggsvk smithi062 running (3m) 58s ago 3m 50.7M - 3.5 e1d6a67b021e dc36997f65c5 2024-03-24T11:20:57.823 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.x smithi139 *:8443,9283 starting - - - - 2024-03-24T11:20:57.823 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.y smithi062 *:9283 running (11m) 58s ago 11m 467M - 17.2.0 e1d6a67b021e 28a8eee27817 2024-03-24T11:20:57.823 INFO:teuthology.orchestra.run.smithi062.stdout:mon.a smithi062 running (11m) 58s ago 11m 50.2M 2048M 17.2.0 e1d6a67b021e 712f01aa9180 2024-03-24T11:20:57.823 INFO:teuthology.orchestra.run.smithi062.stdout:mon.b smithi139 running (9m) 52s ago 9m 43.8M 2048M 17.2.0 e1d6a67b021e 8945cdb4e6aa 2024-03-24T11:20:57.823 INFO:teuthology.orchestra.run.smithi062.stdout:mon.c smithi062 running (10m) 58s ago 10m 40.4M 2048M 17.2.0 e1d6a67b021e 39110a16ab62 2024-03-24T11:20:57.823 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.a smithi062 *:9100 starting - - - - 2024-03-24T11:20:57.823 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.b smithi139 *:9100 running (5m) 52s ago 5m 16.7M - 1dbe0e931976 8921c8c780f7 2024-03-24T11:20:57.823 INFO:teuthology.orchestra.run.smithi062.stdout:osd.0 smithi062 running (9m) 58s ago 9m 50.4M 4096M 17.2.0 e1d6a67b021e 8040b30d7170 2024-03-24T11:20:57.823 INFO:teuthology.orchestra.run.smithi062.stdout:osd.1 smithi062 running (8m) 58s ago 8m 52.2M 4096M 17.2.0 e1d6a67b021e e62d22e5dc29 2024-03-24T11:20:57.823 INFO:teuthology.orchestra.run.smithi062.stdout:osd.2 smithi062 running (8m) 58s ago 8m 46.2M 4096M 17.2.0 e1d6a67b021e 714f550103a7 2024-03-24T11:20:57.824 INFO:teuthology.orchestra.run.smithi062.stdout:osd.3 smithi062 running (8m) 58s ago 8m 49.8M 4096M 17.2.0 e1d6a67b021e 1d2868bd5195 2024-03-24T11:20:57.824 INFO:teuthology.orchestra.run.smithi062.stdout:osd.4 smithi139 running (7m) 52s ago 7m 54.5M 4096M 17.2.0 e1d6a67b021e 7568bfa9e74d 2024-03-24T11:20:57.824 INFO:teuthology.orchestra.run.smithi062.stdout:osd.5 smithi139 running (7m) 52s ago 7m 47.7M 4096M 17.2.0 e1d6a67b021e dd0abd599a14 2024-03-24T11:20:57.824 INFO:teuthology.orchestra.run.smithi062.stdout:osd.6 smithi139 running (7m) 52s ago 7m 48.1M 4096M 17.2.0 e1d6a67b021e a4b1289aca69 2024-03-24T11:20:57.824 INFO:teuthology.orchestra.run.smithi062.stdout:osd.7 smithi139 running (6m) 52s ago 6m 50.0M 4096M 17.2.0 e1d6a67b021e bc7fadbc1384 2024-03-24T11:20:57.824 INFO:teuthology.orchestra.run.smithi062.stdout:prometheus.a smithi139 *:9095 running (4m) 52s ago 5m 67.7M - 514e6a882f6e 3ac3b6a49a6e 2024-03-24T11:20:57.824 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi062.sacpds smithi062 *:8000 running (4m) 58s ago 4m 69.9M - 17.2.0 e1d6a67b021e 89f4ac308e98 2024-03-24T11:20:57.824 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi139.yljesh smithi139 *:8000 running (4m) 52s ago 4m 69.1M - 17.2.0 e1d6a67b021e e189bdc12cae 2024-03-24T11:20:57.824 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi062.xvpxnk smithi062 *:80 running (3m) 58s ago 3m 69.7M - 17.2.0 e1d6a67b021e d1aa74adab44 2024-03-24T11:20:57.824 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi139.pbtrmp smithi139 *:80 running (3m) 52s ago 3m 72.3M - 17.2.0 e1d6a67b021e 7f8e336755d7 2024-03-24T11:20:58.253 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:20:58.253 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": { 2024-03-24T11:20:58.253 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 3 2024-03-24T11:20:58.253 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:20:58.253 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": { 2024-03-24T11:20:58.253 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 1, 2024-03-24T11:20:58.253 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 1 2024-03-24T11:20:58.253 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:20:58.253 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": { 2024-03-24T11:20:58.254 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 8 2024-03-24T11:20:58.254 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:20:58.254 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": {}, 2024-03-24T11:20:58.254 INFO:teuthology.orchestra.run.smithi062.stdout: "rgw": { 2024-03-24T11:20:58.254 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4 2024-03-24T11:20:58.254 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:20:58.254 INFO:teuthology.orchestra.run.smithi062.stdout: "overall": { 2024-03-24T11:20:58.254 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 16, 2024-03-24T11:20:58.254 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 1 2024-03-24T11:20:58.254 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-24T11:20:58.254 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:20:58.622 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:20:58.622 INFO:teuthology.orchestra.run.smithi062.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69", 2024-03-24T11:20:58.622 INFO:teuthology.orchestra.run.smithi062.stdout: "in_progress": true, 2024-03-24T11:20:58.622 INFO:teuthology.orchestra.run.smithi062.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T11:20:58.622 INFO:teuthology.orchestra.run.smithi062.stdout: "services_complete": [], 2024-03-24T11:20:58.622 INFO:teuthology.orchestra.run.smithi062.stdout: "progress": "", 2024-03-24T11:20:58.622 INFO:teuthology.orchestra.run.smithi062.stdout: "message": "", 2024-03-24T11:20:58.623 INFO:teuthology.orchestra.run.smithi062.stdout: "is_paused": false 2024-03-24T11:20:58.623 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:20:59.058 INFO:teuthology.orchestra.run.smithi062.stdout:HEALTH_OK 2024-03-24T11:20:59.090 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:20:58 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[25042]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:20:58] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-03-24T11:20:59.090 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:58 smithi062 ceph-mon[24799]: from='client.15207 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:20:59.090 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:58 smithi062 ceph-mon[24799]: from='client.15213 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:20:59.090 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:58 smithi062 ceph-mon[24799]: from='client.25123 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:20:59.090 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:58 smithi062 ceph-mon[24799]: pgmap v20: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-03-24T11:20:59.091 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:58 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/1712010814' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:20:59.091 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:58 smithi062 ceph-mon[29666]: from='client.15207 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:20:59.091 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:58 smithi062 ceph-mon[29666]: from='client.15213 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:20:59.091 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:58 smithi062 ceph-mon[29666]: from='client.25123 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:20:59.091 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:58 smithi062 ceph-mon[29666]: pgmap v20: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-03-24T11:20:59.091 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:58 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/1712010814' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:20:59.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:58 smithi139 ceph-mon[25719]: from='client.15207 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:20:59.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:58 smithi139 ceph-mon[25719]: from='client.15213 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:20:59.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:58 smithi139 ceph-mon[25719]: from='client.25123 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:20:59.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:58 smithi139 ceph-mon[25719]: pgmap v20: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-03-24T11:20:59.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:58 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/1712010814' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:20:59.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:59 smithi062 ceph-mon[24799]: from='client.25135 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:20:59.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:59 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/4084257118' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:20:59.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:20:59 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/419862848' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-24T11:20:59.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:59 smithi062 ceph-mon[29666]: from='client.25135 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:20:59.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:59 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/4084257118' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:20:59.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:20:59 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/419862848' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-24T11:21:00.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:59 smithi139 ceph-mon[25719]: from='client.25135 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:21:00.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:59 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/4084257118' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:21:00.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:20:59 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/419862848' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-24T11:21:01.108 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:00 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:21:00] "GET /metrics HTTP/1.1" 200 38151 "" "Prometheus/2.33.4" 2024-03-24T11:21:01.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:00 smithi139 ceph-mon[25719]: pgmap v21: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 341 B/s rd, 0 op/s 2024-03-24T11:21:01.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:00 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/1163801708' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/3312461240"}]: dispatch 2024-03-24T11:21:01.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:00 smithi139 ceph-mon[25719]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/3312461240"}]: dispatch 2024-03-24T11:21:01.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:00 smithi062 ceph-mon[24799]: pgmap v21: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 341 B/s rd, 0 op/s 2024-03-24T11:21:01.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:00 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/1163801708' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/3312461240"}]: dispatch 2024-03-24T11:21:01.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:00 smithi062 ceph-mon[24799]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/3312461240"}]: dispatch 2024-03-24T11:21:01.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:00 smithi062 ceph-mon[29666]: pgmap v21: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 341 B/s rd, 0 op/s 2024-03-24T11:21:01.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:00 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/1163801708' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/3312461240"}]: dispatch 2024-03-24T11:21:01.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:00 smithi062 ceph-mon[29666]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/3312461240"}]: dispatch 2024-03-24T11:21:02.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:01 smithi062 ceph-mon[24799]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/3312461240"}]': finished 2024-03-24T11:21:02.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:01 smithi062 ceph-mon[24799]: osdmap e99: 8 total, 8 up, 8 in 2024-03-24T11:21:02.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:01 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/1603759635' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/555071628"}]: dispatch 2024-03-24T11:21:02.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:01 smithi062 ceph-mon[24799]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/555071628"}]: dispatch 2024-03-24T11:21:02.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:01 smithi062 ceph-mon[29666]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/3312461240"}]': finished 2024-03-24T11:21:02.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:01 smithi062 ceph-mon[29666]: osdmap e99: 8 total, 8 up, 8 in 2024-03-24T11:21:02.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:01 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/1603759635' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/555071628"}]: dispatch 2024-03-24T11:21:02.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:01 smithi062 ceph-mon[29666]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/555071628"}]: dispatch 2024-03-24T11:21:02.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:01 smithi139 ceph-mon[25719]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/3312461240"}]': finished 2024-03-24T11:21:02.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:01 smithi139 ceph-mon[25719]: osdmap e99: 8 total, 8 up, 8 in 2024-03-24T11:21:02.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:01 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/1603759635' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/555071628"}]: dispatch 2024-03-24T11:21:02.315 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:01 smithi139 ceph-mon[25719]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/555071628"}]: dispatch 2024-03-24T11:21:02.608 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:02 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[46110]: ts=2024-03-24T11:21:02.314Z caller=manager.go:609 level=warn component="rule manager" group=pools msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on(pool_id) group_right()\n ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: |\n Pool '{{ $labels.name }}' will be full in less than 5 days assuming the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed it's capacity\n" err="found duplicate series for the match group {pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"172.21.15.62:9283\", job=\"ceph\", pool_id=\"1\"}, {instance=\"172.21.15.139:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-24T11:21:03.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:02 smithi062 ceph-mon[24799]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/555071628"}]': finished 2024-03-24T11:21:03.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:02 smithi062 ceph-mon[24799]: osdmap e100: 8 total, 8 up, 8 in 2024-03-24T11:21:03.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:02 smithi062 ceph-mon[24799]: pgmap v24: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 127 B/s rd, 0 op/s 2024-03-24T11:21:03.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:02 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/1218565276' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/3932249987"}]: dispatch 2024-03-24T11:21:03.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:02 smithi062 ceph-mon[29666]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/555071628"}]': finished 2024-03-24T11:21:03.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:02 smithi062 ceph-mon[29666]: osdmap e100: 8 total, 8 up, 8 in 2024-03-24T11:21:03.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:02 smithi062 ceph-mon[29666]: pgmap v24: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 127 B/s rd, 0 op/s 2024-03-24T11:21:03.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:02 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/1218565276' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/3932249987"}]: dispatch 2024-03-24T11:21:03.227 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:02 smithi139 ceph-mon[25719]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/555071628"}]': finished 2024-03-24T11:21:03.228 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:02 smithi139 ceph-mon[25719]: osdmap e100: 8 total, 8 up, 8 in 2024-03-24T11:21:03.228 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:02 smithi139 ceph-mon[25719]: pgmap v24: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 127 B/s rd, 0 op/s 2024-03-24T11:21:03.228 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:02 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/1218565276' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/3932249987"}]: dispatch 2024-03-24T11:21:04.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:03 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/1218565276' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/3932249987"}]': finished 2024-03-24T11:21:04.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:03 smithi062 ceph-mon[24799]: osdmap e101: 8 total, 8 up, 8 in 2024-03-24T11:21:04.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:03 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/1869030073' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/793412542"}]: dispatch 2024-03-24T11:21:04.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:03 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/1218565276' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/3932249987"}]': finished 2024-03-24T11:21:04.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:03 smithi062 ceph-mon[29666]: osdmap e101: 8 total, 8 up, 8 in 2024-03-24T11:21:04.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:03 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/1869030073' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/793412542"}]: dispatch 2024-03-24T11:21:04.351 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:03 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/1218565276' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/3932249987"}]': finished 2024-03-24T11:21:04.351 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:03 smithi139 ceph-mon[25719]: osdmap e101: 8 total, 8 up, 8 in 2024-03-24T11:21:04.351 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:03 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/1869030073' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/793412542"}]: dispatch 2024-03-24T11:21:04.608 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:04 smithi139 systemd[1]: Stopping Ceph prometheus.a for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:21:04.608 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:04 smithi139 bash[55529]: Error: no container with name or ID "ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus.a" found: no such container 2024-03-24T11:21:04.929 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:04 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[46110]: ts=2024-03-24T11:21:04.643Z caller=main.go:775 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-03-24T11:21:04.929 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:04 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[46110]: ts=2024-03-24T11:21:04.643Z caller=main.go:798 level=info msg="Stopping scrape discovery manager..." 2024-03-24T11:21:04.929 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:04 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[46110]: ts=2024-03-24T11:21:04.643Z caller=main.go:812 level=info msg="Stopping notify discovery manager..." 2024-03-24T11:21:04.929 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:04 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[46110]: ts=2024-03-24T11:21:04.643Z caller=main.go:834 level=info msg="Stopping scrape manager..." 2024-03-24T11:21:04.929 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:04 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[46110]: ts=2024-03-24T11:21:04.643Z caller=main.go:808 level=info msg="Notify discovery manager stopped" 2024-03-24T11:21:04.929 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:04 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[46110]: ts=2024-03-24T11:21:04.643Z caller=main.go:794 level=info msg="Scrape discovery manager stopped" 2024-03-24T11:21:04.929 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:04 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[46110]: ts=2024-03-24T11:21:04.644Z caller=manager.go:945 level=info component="rule manager" msg="Stopping rule manager..." 2024-03-24T11:21:04.930 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:04 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[46110]: ts=2024-03-24T11:21:04.644Z caller=main.go:828 level=info msg="Scrape manager stopped" 2024-03-24T11:21:04.930 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:04 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[46110]: ts=2024-03-24T11:21:04.644Z caller=manager.go:955 level=info component="rule manager" msg="Rule manager stopped" 2024-03-24T11:21:04.930 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:04 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[46110]: ts=2024-03-24T11:21:04.644Z caller=notifier.go:600 level=info component=notifier msg="Stopping notification manager..." 2024-03-24T11:21:04.930 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:04 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[46110]: ts=2024-03-24T11:21:04.644Z caller=main.go:1054 level=info msg="Notifier manager stopped" 2024-03-24T11:21:04.930 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:04 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[46110]: ts=2024-03-24T11:21:04.644Z caller=main.go:1066 level=info msg="See you next time!" 2024-03-24T11:21:04.930 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:04 smithi139 podman[55540]: 2024-03-24 11:21:04.654135697 +0000 UTC m=+0.127907853 container died 3ac3b6a49a6ed3d04961e96124562a615ed65db9e2990c48c93bae547683b1a3 (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-24T11:21:05.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:04 smithi062 ceph-mon[24799]: pgmap v26: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-03-24T11:21:05.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:04 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/1869030073' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/793412542"}]': finished 2024-03-24T11:21:05.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:04 smithi062 ceph-mon[24799]: osdmap e102: 8 total, 8 up, 8 in 2024-03-24T11:21:05.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:04 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/3813594313' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/2398349393"}]: dispatch 2024-03-24T11:21:05.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:04 smithi062 ceph-mon[24799]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/2398349393"}]: dispatch 2024-03-24T11:21:05.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:04 smithi062 ceph-mon[29666]: pgmap v26: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-03-24T11:21:05.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:04 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/1869030073' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/793412542"}]': finished 2024-03-24T11:21:05.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:04 smithi062 ceph-mon[29666]: osdmap e102: 8 total, 8 up, 8 in 2024-03-24T11:21:05.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:04 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/3813594313' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/2398349393"}]: dispatch 2024-03-24T11:21:05.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:04 smithi062 ceph-mon[29666]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/2398349393"}]: dispatch 2024-03-24T11:21:05.297 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:05 smithi139 podman[55540]: 2024-03-24 11:21:05.046871181 +0000 UTC m=+0.520643345 container cleanup 3ac3b6a49a6ed3d04961e96124562a615ed65db9e2990c48c93bae547683b1a3 (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-24T11:21:05.297 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:05 smithi139 bash[55540]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a 2024-03-24T11:21:05.297 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:05 smithi139 bash[55567]: Error: no container with name or ID "ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus.a" found: no such container 2024-03-24T11:21:05.298 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:04 smithi139 ceph-mon[25719]: pgmap v26: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-03-24T11:21:05.298 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:04 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/1869030073' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/793412542"}]': finished 2024-03-24T11:21:05.298 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:04 smithi139 ceph-mon[25719]: osdmap e102: 8 total, 8 up, 8 in 2024-03-24T11:21:05.298 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:04 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/3813594313' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/2398349393"}]: dispatch 2024-03-24T11:21:05.298 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:04 smithi139 ceph-mon[25719]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/2398349393"}]: dispatch 2024-03-24T11:21:05.556 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:05 smithi139 podman[55554]: 2024-03-24 11:21:05.297828845 +0000 UTC m=+0.646223946 container remove 3ac3b6a49a6ed3d04961e96124562a615ed65db9e2990c48c93bae547683b1a3 (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-24T11:21:05.557 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:05 smithi139 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@prometheus.a.service: Deactivated successfully. 2024-03-24T11:21:05.557 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:05 smithi139 systemd[1]: Stopped Ceph prometheus.a for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:21:05.557 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:05 smithi139 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@prometheus.a.service: Consumed 1.515s CPU time. 2024-03-24T11:21:05.557 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:05 smithi139 systemd[1]: Starting Ceph prometheus.a for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:21:05.557 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:05 smithi139 podman[55642]: 2024-03-24 11:21:05.436305349 +0000 UTC m=+0.019136281 image pull 514e6a882f6e74806a5856468489eeff8d7106095557578da96935e4d0ba4d9d quay.io/prometheus/prometheus:v2.33.4 2024-03-24T11:21:05.557 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:05 smithi139 podman[55642]: 2024-03-24 11:21:05.557226074 +0000 UTC m=+0.140057002 container create 5c0c71f4d276d17a5bbb8f08f35a21485368788f3a16842bc4346b323bfe6c33 (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-24T11:21:06.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:05 smithi139 ceph-mon[25719]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/2398349393"}]': finished 2024-03-24T11:21:06.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:05 smithi139 ceph-mon[25719]: osdmap e103: 8 total, 8 up, 8 in 2024-03-24T11:21:06.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:05 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/666717641' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6801/1600445979"}]: dispatch 2024-03-24T11:21:06.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:05 smithi139 ceph-mon[25719]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6801/1600445979"}]: dispatch 2024-03-24T11:21:06.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:05 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:06.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:05 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:06.108 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:05 smithi139 podman[55642]: 2024-03-24 11:21:05.809850363 +0000 UTC m=+0.392681291 container init 5c0c71f4d276d17a5bbb8f08f35a21485368788f3a16842bc4346b323bfe6c33 (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-24T11:21:06.108 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:05 smithi139 podman[55642]: 2024-03-24 11:21:05.81241129 +0000 UTC m=+0.395242219 container start 5c0c71f4d276d17a5bbb8f08f35a21485368788f3a16842bc4346b323bfe6c33 (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-24T11:21:06.108 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:05 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[55656]: ts=2024-03-24T11:21:05.836Z caller=main.go:475 level=info msg="No time or size retention was set so using the default time retention" duration=15d 2024-03-24T11:21:06.108 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:05 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[55656]: ts=2024-03-24T11:21:05.836Z caller=main.go:512 level=info msg="Starting Prometheus" version="(version=2.33.4, branch=HEAD, revision=83032011a5d3e6102624fe58241a374a7201fee8)" 2024-03-24T11:21:06.109 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:05 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[55656]: ts=2024-03-24T11:21:05.836Z caller=main.go:517 level=info build_context="(go=go1.17.7, user=root@d13bf69e7be8, date=20220222-16:51:28)" 2024-03-24T11:21:06.109 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:05 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[55656]: ts=2024-03-24T11:21:05.836Z caller=main.go:518 level=info host_details="(Linux 5.14.0-430.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024 x86_64 smithi139 (none))" 2024-03-24T11:21:06.109 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:05 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[55656]: ts=2024-03-24T11:21:05.836Z caller=main.go:519 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-03-24T11:21:06.109 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:05 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[55656]: ts=2024-03-24T11:21:05.836Z caller=main.go:520 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-03-24T11:21:06.109 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:05 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[55656]: ts=2024-03-24T11:21:05.837Z caller=web.go:570 level=info component=web msg="Start listening for connections" address=:9095 2024-03-24T11:21:06.109 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:05 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[55656]: ts=2024-03-24T11:21:05.838Z caller=main.go:923 level=info msg="Starting TSDB ..." 2024-03-24T11:21:06.109 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:05 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[55656]: ts=2024-03-24T11:21:05.839Z caller=tls_config.go:195 level=info component=web msg="TLS is disabled." http2=false 2024-03-24T11:21:06.109 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:05 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[55656]: ts=2024-03-24T11:21:05.841Z caller=head.go:493 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-03-24T11:21:06.109 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:05 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[55656]: ts=2024-03-24T11:21:05.841Z caller=head.go:527 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.789µs 2024-03-24T11:21:06.109 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:05 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[55656]: ts=2024-03-24T11:21:05.841Z caller=head.go:533 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-03-24T11:21:06.109 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:05 smithi139 bash[55642]: 5c0c71f4d276d17a5bbb8f08f35a21485368788f3a16842bc4346b323bfe6c33 2024-03-24T11:21:06.109 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:05 smithi139 systemd[1]: Started Ceph prometheus.a for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:21:06.109 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:06 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[55656]: ts=2024-03-24T11:21:06.032Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=4 2024-03-24T11:21:06.110 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:06 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[55656]: ts=2024-03-24T11:21:06.032Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=4 2024-03-24T11:21:06.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:05 smithi062 ceph-mon[24799]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/2398349393"}]': finished 2024-03-24T11:21:06.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:05 smithi062 ceph-mon[24799]: osdmap e103: 8 total, 8 up, 8 in 2024-03-24T11:21:06.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:05 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/666717641' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6801/1600445979"}]: dispatch 2024-03-24T11:21:06.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:05 smithi062 ceph-mon[24799]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6801/1600445979"}]: dispatch 2024-03-24T11:21:06.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:05 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:06.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:05 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:06.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:05 smithi062 ceph-mon[29666]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/2398349393"}]': finished 2024-03-24T11:21:06.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:05 smithi062 ceph-mon[29666]: osdmap e103: 8 total, 8 up, 8 in 2024-03-24T11:21:06.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:05 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/666717641' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6801/1600445979"}]: dispatch 2024-03-24T11:21:06.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:05 smithi062 ceph-mon[29666]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6801/1600445979"}]: dispatch 2024-03-24T11:21:06.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:05 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:06.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:05 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:06.514 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:06 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[55656]: ts=2024-03-24T11:21:06.219Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=2 maxSegment=4 2024-03-24T11:21:06.514 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:06 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[55656]: ts=2024-03-24T11:21:06.316Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=3 maxSegment=4 2024-03-24T11:21:06.514 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:06 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[55656]: ts=2024-03-24T11:21:06.316Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=4 maxSegment=4 2024-03-24T11:21:06.514 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:06 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[55656]: ts=2024-03-24T11:21:06.316Z caller=head.go:610 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=36.316µs wal_replay_duration=475.072933ms total_replay_duration=475.123964ms 2024-03-24T11:21:06.514 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:06 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[55656]: ts=2024-03-24T11:21:06.320Z caller=main.go:944 level=info fs_type=XFS_SUPER_MAGIC 2024-03-24T11:21:06.515 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:06 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[55656]: ts=2024-03-24T11:21:06.320Z caller=main.go:947 level=info msg="TSDB started" 2024-03-24T11:21:06.515 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:06 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[55656]: ts=2024-03-24T11:21:06.320Z caller=main.go:1128 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-03-24T11:21:06.515 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:06 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[55656]: ts=2024-03-24T11:21:06.341Z caller=main.go:1165 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=21.812339ms db_storage=780ns remote_storage=1.195µs web_handler=386ns query_engine=769ns scrape=537.218µs scrape_sd=86.522µs notify=25.024µs notify_sd=26.482µs rules=20.835541ms 2024-03-24T11:21:06.515 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:06 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[55656]: ts=2024-03-24T11:21:06.341Z caller=main.go:896 level=info msg="Server is ready to receive web requests." 2024-03-24T11:21:07.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:06 smithi062 ceph-mon[24799]: pgmap v29: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:21:07.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:06 smithi062 ceph-mon[24799]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6801/1600445979"}]': finished 2024-03-24T11:21:07.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:06 smithi062 ceph-mon[24799]: osdmap e104: 8 total, 8 up, 8 in 2024-03-24T11:21:07.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:06 smithi062 ceph-mon[24799]: Reconfiguring node-exporter.b (dependencies changed)... 2024-03-24T11:21:07.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:06 smithi062 ceph-mon[24799]: Reconfiguring daemon node-exporter.b on smithi139 2024-03-24T11:21:07.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:06 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/1459573152' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6800/1600445979"}]: dispatch 2024-03-24T11:21:07.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:06 smithi062 ceph-mon[29666]: pgmap v29: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:21:07.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:06 smithi062 ceph-mon[29666]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6801/1600445979"}]': finished 2024-03-24T11:21:07.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:06 smithi062 ceph-mon[29666]: osdmap e104: 8 total, 8 up, 8 in 2024-03-24T11:21:07.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:06 smithi062 ceph-mon[29666]: Reconfiguring node-exporter.b (dependencies changed)... 2024-03-24T11:21:07.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:06 smithi062 ceph-mon[29666]: Reconfiguring daemon node-exporter.b on smithi139 2024-03-24T11:21:07.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:06 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/1459573152' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6800/1600445979"}]: dispatch 2024-03-24T11:21:07.269 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:06 smithi139 ceph-mon[25719]: pgmap v29: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:21:07.269 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:06 smithi139 ceph-mon[25719]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6801/1600445979"}]': finished 2024-03-24T11:21:07.269 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:06 smithi139 ceph-mon[25719]: osdmap e104: 8 total, 8 up, 8 in 2024-03-24T11:21:07.269 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:06 smithi139 ceph-mon[25719]: Reconfiguring node-exporter.b (dependencies changed)... 2024-03-24T11:21:07.269 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:06 smithi139 ceph-mon[25719]: Reconfiguring daemon node-exporter.b on smithi139 2024-03-24T11:21:07.269 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:06 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/1459573152' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6800/1600445979"}]: dispatch 2024-03-24T11:21:08.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:07 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/1459573152' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6800/1600445979"}]': finished 2024-03-24T11:21:08.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:07 smithi139 ceph-mon[25719]: osdmap e105: 8 total, 8 up, 8 in 2024-03-24T11:21:08.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:07 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/1459573152' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6800/1600445979"}]': finished 2024-03-24T11:21:08.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:07 smithi062 ceph-mon[24799]: osdmap e105: 8 total, 8 up, 8 in 2024-03-24T11:21:08.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:07 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/1459573152' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6800/1600445979"}]': finished 2024-03-24T11:21:08.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:07 smithi062 ceph-mon[29666]: osdmap e105: 8 total, 8 up, 8 in 2024-03-24T11:21:09.221 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:08 smithi139 ceph-mon[25719]: pgmap v32: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:21:09.355 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:08 smithi062 ceph-mon[24799]: pgmap v32: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:21:09.355 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:08 smithi062 ceph-mon[29666]: pgmap v32: 161 pgs: 161 active+clean; 585 KiB data, 95 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:21:09.476 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:09 smithi139 systemd[1]: Stopping Ceph node-exporter.b for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:21:09.476 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:09 smithi139 bash[55873]: Error: no container with name or ID "ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter.b" found: no such container 2024-03-24T11:21:09.750 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:09 smithi139 podman[55886]: 2024-03-24 11:21:09.476464615 +0000 UTC m=+0.129355240 container died 8921c8c780f7c30d7013b943c506bc0719456cf780f3d33a796da7e4450c05f8 (image=quay.io/prometheus/node-exporter:v1.3.1, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-03-24T11:21:10.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:09 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:21:10.108 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:09 smithi139 podman[55886]: 2024-03-24 11:21:09.849416117 +0000 UTC m=+0.502306739 container cleanup 8921c8c780f7c30d7013b943c506bc0719456cf780f3d33a796da7e4450c05f8 (image=quay.io/prometheus/node-exporter:v1.3.1, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-03-24T11:21:10.108 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:09 smithi139 bash[55886]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b 2024-03-24T11:21:10.108 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:09 smithi139 bash[55912]: Error: no container with name or ID "ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter.b" found: no such container 2024-03-24T11:21:10.313 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:09 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:21:10.314 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:09 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:21:10.460 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 podman[55900]: 2024-03-24 11:21:10.134430648 +0000 UTC m=+0.660565711 container remove 8921c8c780f7c30d7013b943c506bc0719456cf780f3d33a796da7e4450c05f8 (image=quay.io/prometheus/node-exporter:v1.3.1, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-03-24T11:21:10.461 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-03-24T11:21:10.461 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@node-exporter.b.service: Failed with result 'exit-code'. 2024-03-24T11:21:10.461 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 systemd[1]: Stopped Ceph node-exporter.b for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:21:10.461 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@node-exporter.b.service: Consumed 2.956s CPU time. 2024-03-24T11:21:10.461 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 systemd[1]: Starting Ceph node-exporter.b for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:21:10.461 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 podman[55978]: 2024-03-24 11:21:10.360092444 +0000 UTC m=+0.110359185 container create 8f9f4608e465db9cbd71fcc64ec2dca68e65232f12362b1e4e87818be67c6ced (image=quay.io/prometheus/node-exporter:v1.3.1, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-03-24T11:21:10.461 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 podman[55978]: 2024-03-24 11:21:10.267765649 +0000 UTC m=+0.018032394 image pull 1dbe0e931976487e20e5cfb272087e08a9779c88fd5e9617ed7042dd9751ec26 quay.io/prometheus/node-exporter:v1.3.1 2024-03-24T11:21:10.858 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 podman[55978]: 2024-03-24 11:21:10.54568971 +0000 UTC m=+0.295956451 container init 8f9f4608e465db9cbd71fcc64ec2dca68e65232f12362b1e4e87818be67c6ced (image=quay.io/prometheus/node-exporter:v1.3.1, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-03-24T11:21:10.858 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 podman[55978]: 2024-03-24 11:21:10.548158549 +0000 UTC m=+0.298425292 container start 8f9f4608e465db9cbd71fcc64ec2dca68e65232f12362b1e4e87818be67c6ced (image=quay.io/prometheus/node-exporter:v1.3.1, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-03-24T11:21:10.858 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[55993]: ts=2024-03-24T11:21:10.551Z caller=node_exporter.go:182 level=info msg="Starting node_exporter" version="(version=1.3.1, branch=HEAD, revision=a2321e7b940ddcff26873612bccdf7cd4c42b6b6)" 2024-03-24T11:21:10.858 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[55993]: ts=2024-03-24T11:21:10.551Z caller=node_exporter.go:183 level=info msg="Build context" build_context="(go=go1.17.3, user=root@243aafa5525c, date=20211205-11:09:49)" 2024-03-24T11:21:10.858 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[55993]: ts=2024-03-24T11:21:10.552Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+)($|/) 2024-03-24T11:21:10.858 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[55993]: ts=2024-03-24T11:21:10.552Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-03-24T11:21:10.858 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[55993]: ts=2024-03-24T11:21:10.553Z caller=node_exporter.go:108 level=info msg="Enabled collectors" 2024-03-24T11:21:10.858 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[55993]: ts=2024-03-24T11:21:10.553Z caller=node_exporter.go:115 level=info collector=arp 2024-03-24T11:21:10.859 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[55993]: ts=2024-03-24T11:21:10.553Z caller=node_exporter.go:115 level=info collector=bcache 2024-03-24T11:21:10.859 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[55993]: ts=2024-03-24T11:21:10.553Z caller=node_exporter.go:115 level=info collector=bonding 2024-03-24T11:21:10.859 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[55993]: ts=2024-03-24T11:21:10.553Z caller=node_exporter.go:115 level=info collector=btrfs 2024-03-24T11:21:10.859 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[55993]: ts=2024-03-24T11:21:10.553Z caller=node_exporter.go:115 level=info collector=conntrack 2024-03-24T11:21:10.859 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[55993]: ts=2024-03-24T11:21:10.553Z caller=node_exporter.go:115 level=info collector=cpu 2024-03-24T11:21:10.859 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[55993]: ts=2024-03-24T11:21:10.553Z caller=node_exporter.go:115 level=info collector=cpufreq 2024-03-24T11:21:10.859 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[55993]: ts=2024-03-24T11:21:10.553Z caller=node_exporter.go:115 level=info collector=diskstats 2024-03-24T11:21:10.859 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[55993]: ts=2024-03-24T11:21:10.553Z caller=node_exporter.go:115 level=info collector=dmi 2024-03-24T11:21:10.859 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[55993]: ts=2024-03-24T11:21:10.553Z caller=node_exporter.go:115 level=info collector=edac 2024-03-24T11:21:10.860 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[55993]: ts=2024-03-24T11:21:10.553Z caller=node_exporter.go:115 level=info collector=entropy 2024-03-24T11:21:10.860 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[55993]: ts=2024-03-24T11:21:10.553Z caller=node_exporter.go:115 level=info collector=fibrechannel 2024-03-24T11:21:10.860 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[55993]: ts=2024-03-24T11:21:10.553Z caller=node_exporter.go:115 level=info collector=filefd 2024-03-24T11:21:10.860 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[55993]: ts=2024-03-24T11:21:10.553Z caller=node_exporter.go:115 level=info collector=filesystem 2024-03-24T11:21:10.860 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[55993]: ts=2024-03-24T11:21:10.553Z caller=node_exporter.go:115 level=info collector=hwmon 2024-03-24T11:21:10.860 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[55993]: ts=2024-03-24T11:21:10.553Z caller=node_exporter.go:115 level=info collector=infiniband 2024-03-24T11:21:10.860 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[55993]: ts=2024-03-24T11:21:10.553Z caller=node_exporter.go:115 level=info collector=ipvs 2024-03-24T11:21:10.860 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[55993]: ts=2024-03-24T11:21:10.553Z caller=node_exporter.go:115 level=info collector=loadavg 2024-03-24T11:21:10.860 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[55993]: ts=2024-03-24T11:21:10.553Z caller=node_exporter.go:115 level=info collector=mdadm 2024-03-24T11:21:10.860 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[55993]: ts=2024-03-24T11:21:10.553Z caller=node_exporter.go:115 level=info collector=meminfo 2024-03-24T11:21:10.860 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[55993]: ts=2024-03-24T11:21:10.553Z caller=node_exporter.go:115 level=info collector=netclass 2024-03-24T11:21:10.860 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[55993]: ts=2024-03-24T11:21:10.553Z caller=node_exporter.go:115 level=info collector=netdev 2024-03-24T11:21:10.861 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[55993]: ts=2024-03-24T11:21:10.553Z caller=node_exporter.go:115 level=info collector=netstat 2024-03-24T11:21:10.861 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[55993]: ts=2024-03-24T11:21:10.553Z caller=node_exporter.go:115 level=info collector=nfs 2024-03-24T11:21:10.861 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[55993]: ts=2024-03-24T11:21:10.553Z caller=node_exporter.go:115 level=info collector=nfsd 2024-03-24T11:21:10.861 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[55993]: ts=2024-03-24T11:21:10.553Z caller=node_exporter.go:115 level=info collector=nvme 2024-03-24T11:21:10.861 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[55993]: ts=2024-03-24T11:21:10.553Z caller=node_exporter.go:115 level=info collector=os 2024-03-24T11:21:10.861 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[55993]: ts=2024-03-24T11:21:10.553Z caller=node_exporter.go:115 level=info collector=powersupplyclass 2024-03-24T11:21:10.861 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[55993]: ts=2024-03-24T11:21:10.553Z caller=node_exporter.go:115 level=info collector=pressure 2024-03-24T11:21:10.861 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[55993]: ts=2024-03-24T11:21:10.553Z caller=node_exporter.go:115 level=info collector=rapl 2024-03-24T11:21:10.861 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[55993]: ts=2024-03-24T11:21:10.553Z caller=node_exporter.go:115 level=info collector=schedstat 2024-03-24T11:21:10.861 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[55993]: ts=2024-03-24T11:21:10.553Z caller=node_exporter.go:115 level=info collector=sockstat 2024-03-24T11:21:10.861 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[55993]: ts=2024-03-24T11:21:10.553Z caller=node_exporter.go:115 level=info collector=softnet 2024-03-24T11:21:10.861 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[55993]: ts=2024-03-24T11:21:10.553Z caller=node_exporter.go:115 level=info collector=stat 2024-03-24T11:21:10.861 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[55993]: ts=2024-03-24T11:21:10.553Z caller=node_exporter.go:115 level=info collector=tapestats 2024-03-24T11:21:10.861 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[55993]: ts=2024-03-24T11:21:10.553Z caller=node_exporter.go:115 level=info collector=textfile 2024-03-24T11:21:10.862 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[55993]: ts=2024-03-24T11:21:10.553Z caller=node_exporter.go:115 level=info collector=thermal_zone 2024-03-24T11:21:10.862 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[55993]: ts=2024-03-24T11:21:10.553Z caller=node_exporter.go:115 level=info collector=time 2024-03-24T11:21:10.862 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[55993]: ts=2024-03-24T11:21:10.553Z caller=node_exporter.go:115 level=info collector=udp_queues 2024-03-24T11:21:10.862 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[55993]: ts=2024-03-24T11:21:10.553Z caller=node_exporter.go:115 level=info collector=uname 2024-03-24T11:21:10.862 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[55993]: ts=2024-03-24T11:21:10.553Z caller=node_exporter.go:115 level=info collector=vmstat 2024-03-24T11:21:10.862 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[55993]: ts=2024-03-24T11:21:10.553Z caller=node_exporter.go:115 level=info collector=xfs 2024-03-24T11:21:10.862 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[55993]: ts=2024-03-24T11:21:10.553Z caller=node_exporter.go:115 level=info collector=zfs 2024-03-24T11:21:10.862 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[55993]: ts=2024-03-24T11:21:10.553Z caller=node_exporter.go:199 level=info msg="Listening on" address=:9100 2024-03-24T11:21:10.862 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b[55993]: ts=2024-03-24T11:21:10.553Z caller=tls_config.go:195 level=info msg="TLS is disabled." http2=false 2024-03-24T11:21:10.862 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 bash[55978]: 8f9f4608e465db9cbd71fcc64ec2dca68e65232f12362b1e4e87818be67c6ced 2024-03-24T11:21:10.862 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:21:10 smithi139 systemd[1]: Started Ceph node-exporter.b for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:21:11.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:10 smithi062 ceph-mon[24799]: from='client.25156 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:21:11.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:10 smithi062 ceph-mon[24799]: pgmap v33: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-24T11:21:11.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:10 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:11.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:10 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:11.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:10 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:11.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:10 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:11.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:10 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-24T11:21:11.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:10 smithi062 ceph-mon[29666]: from='client.25156 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:21:11.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:10 smithi062 ceph-mon[29666]: pgmap v33: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-24T11:21:11.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:10 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:11.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:10 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:11.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:10 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:11.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:10 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:11.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:10 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-24T11:21:11.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-mon[25719]: from='client.25156 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:21:11.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-mon[25719]: pgmap v33: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-24T11:21:11.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:11.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:11.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:11.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:11.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:10 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-24T11:21:12.276 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:11 smithi062 ceph-mon[24799]: Reconfiguring grafana.a (dependencies changed)... 2024-03-24T11:21:12.276 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:11 smithi062 ceph-mon[24799]: Regenerating cephadm self-signed grafana TLS certificates 2024-03-24T11:21:12.277 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:11 smithi062 ceph-mon[24799]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-24T11:21:12.277 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:12 smithi062 ceph-mon[24799]: Reconfiguring daemon grafana.a on smithi139 2024-03-24T11:21:12.277 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:12 smithi062 ceph-mon[29666]: Reconfiguring grafana.a (dependencies changed)... 2024-03-24T11:21:12.277 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:12 smithi062 ceph-mon[29666]: Regenerating cephadm self-signed grafana TLS certificates 2024-03-24T11:21:12.277 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:12 smithi062 ceph-mon[29666]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-24T11:21:12.277 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:12 smithi062 ceph-mon[29666]: Reconfiguring daemon grafana.a on smithi139 2024-03-24T11:21:12.348 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:12 smithi139 ceph-mon[25719]: Reconfiguring grafana.a (dependencies changed)... 2024-03-24T11:21:12.349 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:12 smithi139 ceph-mon[25719]: Regenerating cephadm self-signed grafana TLS certificates 2024-03-24T11:21:12.349 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:12 smithi139 ceph-mon[25719]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-24T11:21:12.349 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:12 smithi139 ceph-mon[25719]: Reconfiguring daemon grafana.a on smithi139 2024-03-24T11:21:13.303 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:13 smithi139 ceph-mon[25719]: pgmap v34: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 733 B/s rd, 0 op/s 2024-03-24T11:21:13.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:13 smithi062 ceph-mon[24799]: pgmap v34: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 733 B/s rd, 0 op/s 2024-03-24T11:21:13.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:13 smithi062 ceph-mon[29666]: pgmap v34: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 733 B/s rd, 0 op/s 2024-03-24T11:21:15.298 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:15 smithi062 ceph-mon[24799]: pgmap v35: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-24T11:21:15.298 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:15 smithi062 ceph-mon[29666]: pgmap v35: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-24T11:21:15.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:15 smithi139 ceph-mon[25719]: pgmap v35: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-24T11:21:17.348 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:17 smithi139 ceph-mon[25719]: pgmap v36: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-24T11:21:17.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:17 smithi062 ceph-mon[24799]: pgmap v36: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-24T11:21:17.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:17 smithi062 ceph-mon[29666]: pgmap v36: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-24T11:21:19.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:18 smithi062 ceph-mon[24799]: pgmap v37: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 933 B/s rd, 0 op/s 2024-03-24T11:21:19.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:18 smithi062 ceph-mon[29666]: pgmap v37: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 933 B/s rd, 0 op/s 2024-03-24T11:21:19.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:18 smithi139 ceph-mon[25719]: pgmap v37: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 933 B/s rd, 0 op/s 2024-03-24T11:21:19.358 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:19 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:21:19] "GET /metrics HTTP/1.1" 200 38150 "" "Prometheus/2.33.4" 2024-03-24T11:21:21.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:20 smithi139 ceph-mon[25719]: from='client.25156 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:21:21.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:20 smithi139 ceph-mon[25719]: pgmap v38: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:21:21.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:20 smithi062 ceph-mon[24799]: from='client.25156 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:21:21.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:20 smithi062 ceph-mon[24799]: pgmap v38: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:21:21.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:20 smithi062 ceph-mon[29666]: from='client.25156 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:21:21.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:20 smithi062 ceph-mon[29666]: pgmap v38: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:21:23.348 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:23 smithi139 ceph-mon[25719]: pgmap v39: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:21:23.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:23 smithi062 ceph-mon[24799]: pgmap v39: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:21:23.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:23 smithi062 ceph-mon[29666]: pgmap v39: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:21:24.608 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:21:24 smithi139 systemd[1]: Stopping Ceph grafana.a for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:21:24.608 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:21:24 smithi139 bash[56822]: Error: no container with name or ID "ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana.a" found: no such container 2024-03-24T11:21:24.608 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:21:24 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[45103]: t=2024-03-24T11:21:24+0000 lvl=info msg="Shutdown started" logger=server reason="System signal: terminated" 2024-03-24T11:21:24.608 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:21:24 smithi139 podman[56835]: 2024-03-24 11:21:24.538231697 +0000 UTC m=+0.116650351 container died a0bc435a222cd86caa9a4d829524320c2f802078e31078a7aa27feff45311d89 (image=quay.io/ceph/ceph-grafana:8.3.5, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, name=ubi8, maintainer=Paul Cuzner , vendor=Red Hat, Inc., build-date=2022-03-28T10:36:18.413762, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-236.1648460182, com.redhat.component=ubi8-container, distribution-scope=public, io.openshift.expose-services=, version=8.5, release=236.1648460182, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., summary=Grafana Container configured for Ceph mgr/dashboard integration, architecture=x86_64, io.k8s.display-name=Red Hat Universal Base Image 8, io.openshift.tags=base rhel8, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, vcs-type=git, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.buildah.version=1.24.2, description=Ceph Grafana Container) 2024-03-24T11:21:25.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:25 smithi139 ceph-mon[25719]: pgmap v40: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:21:25.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:25 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:21:25.108 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:21:24 smithi139 podman[56835]: 2024-03-24 11:21:24.869045734 +0000 UTC m=+0.447464387 container cleanup a0bc435a222cd86caa9a4d829524320c2f802078e31078a7aa27feff45311d89 (image=quay.io/ceph/ceph-grafana:8.3.5, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a, maintainer=Paul Cuzner , summary=Grafana Container configured for Ceph mgr/dashboard integration, vcs-type=git, io.k8s.display-name=Red Hat Universal Base Image 8, distribution-scope=public, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, vendor=Red Hat, Inc., com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, name=ubi8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-236.1648460182, build-date=2022-03-28T10:36:18.413762, release=236.1648460182, io.openshift.tags=base rhel8, io.openshift.expose-services=, io.buildah.version=1.24.2, architecture=x86_64, com.redhat.component=ubi8-container, description=Ceph Grafana Container, version=8.5) 2024-03-24T11:21:25.108 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:21:24 smithi139 bash[56835]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a 2024-03-24T11:21:25.279 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:25 smithi062 ceph-mon[24799]: pgmap v40: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:21:25.279 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:25 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:21:25.279 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:25 smithi062 ceph-mon[29666]: pgmap v40: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:21:25.279 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:25 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:21:25.380 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:21:25 smithi139 podman[56849]: 2024-03-24 11:21:25.112434615 +0000 UTC m=+0.573589376 container remove a0bc435a222cd86caa9a4d829524320c2f802078e31078a7aa27feff45311d89 (image=quay.io/ceph/ceph-grafana:8.3.5, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vcs-type=git, architecture=x86_64, build-date=2022-03-28T10:36:18.413762, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, vendor=Red Hat, Inc., com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, io.k8s.display-name=Red Hat Universal Base Image 8, name=ubi8, io.openshift.expose-services=, io.buildah.version=1.24.2, distribution-scope=public, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, com.redhat.component=ubi8-container, release=236.1648460182, io.openshift.tags=base rhel8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-236.1648460182, description=Ceph Grafana Container, summary=Grafana Container configured for Ceph mgr/dashboard integration, maintainer=Paul Cuzner , version=8.5) 2024-03-24T11:21:25.380 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:21:25 smithi139 bash[56861]: Error: no container with name or ID "ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana.a" found: no such container 2024-03-24T11:21:25.380 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:21:25 smithi139 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@grafana.a.service: Deactivated successfully. 2024-03-24T11:21:25.380 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:21:25 smithi139 systemd[1]: Stopped Ceph grafana.a for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:21:25.380 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:21:25 smithi139 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@grafana.a.service: Consumed 2.571s CPU time. 2024-03-24T11:21:25.380 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:21:25 smithi139 systemd[1]: Starting Ceph grafana.a for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:21:25.380 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:21:25 smithi139 podman[56930]: 2024-03-24 11:21:25.261491344 +0000 UTC m=+0.017184077 image pull dad864ee21e98e69f4029d1e417aa085001566be0d322fbc75bc6f29b0050c01 quay.io/ceph/ceph-grafana:8.3.5 2024-03-24T11:21:25.380 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:21:25 smithi139 podman[56930]: 2024-03-24 11:21:25.380308593 +0000 UTC m=+0.136001310 container create b48167e7171e96ba7723a932d5c8a043cfffda170060995294227dc1c52336b8 (image=quay.io/ceph/ceph-grafana:8.3.5, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a, distribution-scope=public, release=236.1648460182, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-236.1648460182, version=8.5, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, description=Ceph Grafana Container, io.openshift.expose-services=, vcs-type=git, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.k8s.display-name=Red Hat Universal Base Image 8, summary=Grafana Container configured for Ceph mgr/dashboard integration, build-date=2022-03-28T10:36:18.413762, name=ubi8, io.openshift.tags=base rhel8, io.buildah.version=1.24.2, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, maintainer=Paul Cuzner , vendor=Red Hat, Inc., architecture=x86_64, com.redhat.component=ubi8-container) 2024-03-24T11:21:25.891 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: [24/Mar/2024:11:21:25] ENGINE Bus STOPPING 2024-03-24T11:21:25.891 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: [24/Mar/2024:11:21:25] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-03-24T11:21:25.891 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: [24/Mar/2024:11:21:25] ENGINE Bus STOPPED 2024-03-24T11:21:25.891 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: [24/Mar/2024:11:21:25] ENGINE Bus STARTING 2024-03-24T11:21:25.892 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:21:25 smithi139 podman[56930]: 2024-03-24 11:21:25.632769712 +0000 UTC m=+0.388462444 container init b48167e7171e96ba7723a932d5c8a043cfffda170060995294227dc1c52336b8 (image=quay.io/ceph/ceph-grafana:8.3.5, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a, io.openshift.tags=base rhel8, vendor=Red Hat, Inc., io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, maintainer=Paul Cuzner , name=ubi8, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, com.redhat.component=ubi8-container, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-236.1648460182, description=Ceph Grafana Container, distribution-scope=public, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, vcs-type=git, io.k8s.display-name=Red Hat Universal Base Image 8, io.openshift.expose-services=, version=8.5, build-date=2022-03-28T10:36:18.413762, architecture=x86_64, io.buildah.version=1.24.2, summary=Grafana Container configured for Ceph mgr/dashboard integration, release=236.1648460182) 2024-03-24T11:21:25.892 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:21:25 smithi139 podman[56930]: 2024-03-24 11:21:25.635420209 +0000 UTC m=+0.391112936 container start b48167e7171e96ba7723a932d5c8a043cfffda170060995294227dc1c52336b8 (image=quay.io/ceph/ceph-grafana:8.3.5, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.k8s.display-name=Red Hat Universal Base Image 8, version=8.5, io.openshift.tags=base rhel8, vendor=Red Hat, Inc., io.buildah.version=1.24.2, description=Ceph Grafana Container, maintainer=Paul Cuzner , com.redhat.component=ubi8-container, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-236.1648460182, vcs-type=git, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, summary=Grafana Container configured for Ceph mgr/dashboard integration, release=236.1648460182, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, io.openshift.expose-services=, build-date=2022-03-28T10:36:18.413762, distribution-scope=public, name=ubi8, architecture=x86_64) 2024-03-24T11:21:25.892 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:21:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[56945]: t=2024-03-24T11:21:25+0000 lvl=info msg="The state of unified alerting is still not defined. The decision will be made during as we run the database migrations" logger=settings 2024-03-24T11:21:25.892 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:21:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[56945]: t=2024-03-24T11:21:25+0000 lvl=warn msg="falling back to legacy setting of 'min_interval_seconds'; please use the configuration option in the `unified_alerting` section if Grafana 8 alerts are enabled." logger=settings 2024-03-24T11:21:25.892 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:21:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[56945]: t=2024-03-24T11:21:25+0000 lvl=info msg="Config loaded from" logger=settings file=/usr/share/grafana/conf/defaults.ini 2024-03-24T11:21:25.892 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:21:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[56945]: t=2024-03-24T11:21:25+0000 lvl=info msg="Config loaded from" logger=settings file=/etc/grafana/grafana.ini 2024-03-24T11:21:25.892 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:21:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[56945]: t=2024-03-24T11:21:25+0000 lvl=info msg="Config overridden from Environment variable" logger=settings var="GF_PATHS_DATA=/var/lib/grafana" 2024-03-24T11:21:25.892 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:21:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[56945]: t=2024-03-24T11:21:25+0000 lvl=info msg="Config overridden from Environment variable" logger=settings var="GF_PATHS_LOGS=/var/log/grafana" 2024-03-24T11:21:25.892 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:21:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[56945]: t=2024-03-24T11:21:25+0000 lvl=info msg="Config overridden from Environment variable" logger=settings var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-03-24T11:21:25.892 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:21:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[56945]: t=2024-03-24T11:21:25+0000 lvl=info msg="Config overridden from Environment variable" logger=settings var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-03-24T11:21:25.893 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:21:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[56945]: t=2024-03-24T11:21:25+0000 lvl=info msg="Path Home" logger=settings path=/usr/share/grafana 2024-03-24T11:21:25.893 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:21:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[56945]: t=2024-03-24T11:21:25+0000 lvl=info msg="Path Data" logger=settings path=/var/lib/grafana 2024-03-24T11:21:25.893 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:21:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[56945]: t=2024-03-24T11:21:25+0000 lvl=info msg="Path Logs" logger=settings path=/var/log/grafana 2024-03-24T11:21:25.893 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:21:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[56945]: t=2024-03-24T11:21:25+0000 lvl=info msg="Path Plugins" logger=settings path=/var/lib/grafana/plugins 2024-03-24T11:21:25.893 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:21:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[56945]: t=2024-03-24T11:21:25+0000 lvl=info msg="Path Provisioning" logger=settings path=/etc/grafana/provisioning 2024-03-24T11:21:25.893 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:21:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[56945]: t=2024-03-24T11:21:25+0000 lvl=info msg="App mode production" logger=settings 2024-03-24T11:21:25.893 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:21:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[56945]: t=2024-03-24T11:21:25+0000 lvl=info msg="Connecting to DB" logger=sqlstore dbtype=sqlite3 2024-03-24T11:21:25.893 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:21:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[56945]: t=2024-03-24T11:21:25+0000 lvl=warn msg="SQLite database file has broader permissions than it should" logger=sqlstore path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-03-24T11:21:25.893 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:21:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[56945]: t=2024-03-24T11:21:25+0000 lvl=info msg="Starting DB migrations" logger=migrator 2024-03-24T11:21:25.893 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:21:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[56945]: t=2024-03-24T11:21:25+0000 lvl=info msg="migrations completed" logger=migrator performed=0 skipped=377 duration=571.347µs 2024-03-24T11:21:25.893 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:21:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[56945]: t=2024-03-24T11:21:25+0000 lvl=info msg="Created default organization" logger=sqlstore 2024-03-24T11:21:25.893 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:21:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[56945]: t=2024-03-24T11:21:25+0000 lvl=info msg="Initialising plugins" logger=plugin.manager 2024-03-24T11:21:25.893 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:21:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[56945]: t=2024-03-24T11:21:25+0000 lvl=info msg="Plugin registered" logger=plugin.manager pluginId=input 2024-03-24T11:21:25.894 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:21:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[56945]: t=2024-03-24T11:21:25+0000 lvl=info msg="Plugin registered" logger=plugin.manager pluginId=grafana-piechart-panel 2024-03-24T11:21:25.894 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:21:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[56945]: t=2024-03-24T11:21:25+0000 lvl=info msg="Plugin registered" logger=plugin.manager pluginId=vonage-status-panel 2024-03-24T11:21:25.894 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:21:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[56945]: t=2024-03-24T11:21:25+0000 lvl=info msg="Live Push Gateway initialization" logger=live.push_http 2024-03-24T11:21:25.894 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:21:25 smithi139 bash[56930]: b48167e7171e96ba7723a932d5c8a043cfffda170060995294227dc1c52336b8 2024-03-24T11:21:25.894 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:21:25 smithi139 systemd[1]: Started Ceph grafana.a for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:21:25.894 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:21:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[56945]: t=2024-03-24T11:21:25+0000 lvl=info msg="deleted datasource based on configuration" logger=provisioning.datasources name=Dashboard1 2024-03-24T11:21:25.895 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:21:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[56945]: t=2024-03-24T11:21:25+0000 lvl=info msg="inserting datasource from configuration " logger=provisioning.datasources name=Dashboard1 uid=P43CA22E17D0F9596 2024-03-24T11:21:25.895 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:21:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[56945]: t=2024-03-24T11:21:25+0000 lvl=info msg="inserting datasource from configuration " logger=provisioning.datasources name=Loki uid=P8E80F9AEF21F6940 2024-03-24T11:21:25.895 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:21:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[56945]: t=2024-03-24T11:21:25+0000 lvl=info msg="warming cache for startup" logger=ngalert 2024-03-24T11:21:25.895 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:21:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[56945]: t=2024-03-24T11:21:25+0000 lvl=info msg="starting MultiOrg Alertmanager" logger=ngalert.multiorg.alertmanager 2024-03-24T11:21:25.895 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:21:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[56945]: t=2024-03-24T11:21:25+0000 lvl=info msg="HTTP Server Listen" logger=http.server address=[::]:3000 protocol=https subUrl= socket= 2024-03-24T11:21:26.181 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: [24/Mar/2024:11:21:25] ENGINE Serving on http://:::9283 2024-03-24T11:21:26.181 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: [24/Mar/2024:11:21:25] ENGINE Bus STARTED 2024-03-24T11:21:26.182 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: [24/Mar/2024:11:21:25] ENGINE Bus STOPPING 2024-03-24T11:21:26.182 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: [24/Mar/2024:11:21:25] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-03-24T11:21:26.182 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: [24/Mar/2024:11:21:25] ENGINE Bus STOPPED 2024-03-24T11:21:26.182 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: [24/Mar/2024:11:21:25] ENGINE Bus STARTING 2024-03-24T11:21:26.182 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:26 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: [24/Mar/2024:11:21:26] ENGINE Serving on http://:::9283 2024-03-24T11:21:26.182 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:26 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: [24/Mar/2024:11:21:26] ENGINE Bus STARTED 2024-03-24T11:21:26.182 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:26 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: [24/Mar/2024:11:21:26] ENGINE Bus STOPPING 2024-03-24T11:21:26.182 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:26 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: [24/Mar/2024:11:21:26] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-03-24T11:21:26.182 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:26 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: [24/Mar/2024:11:21:26] ENGINE Bus STOPPED 2024-03-24T11:21:26.182 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:26 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: [24/Mar/2024:11:21:26] ENGINE Bus STARTING 2024-03-24T11:21:26.182 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:26 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: [24/Mar/2024:11:21:26] ENGINE Serving on http://:::9283 2024-03-24T11:21:26.182 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:26 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: [24/Mar/2024:11:21:26] ENGINE Bus STARTED 2024-03-24T11:21:26.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:26 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:26.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:26 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:26.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:26 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-24T11:21:26.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:26 smithi062 ceph-mon[24799]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-24T11:21:26.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:26 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi062.front.sepia.ceph.com:9093"}]: dispatch 2024-03-24T11:21:26.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:26 smithi062 ceph-mon[24799]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi062.front.sepia.ceph.com:9093"}]: dispatch 2024-03-24T11:21:26.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:26 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:26.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:26 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T11:21:26.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:26 smithi062 ceph-mon[24799]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T11:21:26.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:26 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi139.front.sepia.ceph.com:9095"}]: dispatch 2024-03-24T11:21:26.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:26 smithi062 ceph-mon[24799]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi139.front.sepia.ceph.com:9095"}]: dispatch 2024-03-24T11:21:26.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:26 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:26.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:26 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-24T11:21:26.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:26 smithi062 ceph-mon[24799]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-24T11:21:26.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:26 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi139.front.sepia.ceph.com:3000"}]: dispatch 2024-03-24T11:21:26.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:26 smithi062 ceph-mon[24799]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi139.front.sepia.ceph.com:3000"}]: dispatch 2024-03-24T11:21:26.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:26 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:26.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:26 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:21:26.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:26 smithi062 ceph-mon[24799]: Upgrade: Need to upgrade myself (mgr.x) 2024-03-24T11:21:26.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:26 smithi062 ceph-mon[24799]: pgmap v41: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:21:26.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:26 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:26.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:26 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:26.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:26 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-24T11:21:26.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:26 smithi062 ceph-mon[29666]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-24T11:21:26.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:26 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi062.front.sepia.ceph.com:9093"}]: dispatch 2024-03-24T11:21:26.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:26 smithi062 ceph-mon[29666]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi062.front.sepia.ceph.com:9093"}]: dispatch 2024-03-24T11:21:26.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:26 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:26.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:26 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T11:21:26.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:26 smithi062 ceph-mon[29666]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T11:21:26.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:26 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi139.front.sepia.ceph.com:9095"}]: dispatch 2024-03-24T11:21:26.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:26 smithi062 ceph-mon[29666]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi139.front.sepia.ceph.com:9095"}]: dispatch 2024-03-24T11:21:26.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:26 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:26.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:26 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-24T11:21:26.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:26 smithi062 ceph-mon[29666]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-24T11:21:26.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:26 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi139.front.sepia.ceph.com:3000"}]: dispatch 2024-03-24T11:21:26.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:26 smithi062 ceph-mon[29666]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi139.front.sepia.ceph.com:3000"}]: dispatch 2024-03-24T11:21:26.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:26 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:26.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:26 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:21:26.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:26 smithi062 ceph-mon[29666]: Upgrade: Need to upgrade myself (mgr.x) 2024-03-24T11:21:26.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:26 smithi062 ceph-mon[29666]: pgmap v41: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:21:27.025 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:26 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:27.026 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:26 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:27.026 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:26 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-24T11:21:27.026 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:26 smithi139 ceph-mon[25719]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-24T11:21:27.026 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:26 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi062.front.sepia.ceph.com:9093"}]: dispatch 2024-03-24T11:21:27.026 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:26 smithi139 ceph-mon[25719]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi062.front.sepia.ceph.com:9093"}]: dispatch 2024-03-24T11:21:27.026 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:26 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:27.026 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:26 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T11:21:27.026 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:26 smithi139 ceph-mon[25719]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T11:21:27.026 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:26 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi139.front.sepia.ceph.com:9095"}]: dispatch 2024-03-24T11:21:27.026 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:26 smithi139 ceph-mon[25719]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi139.front.sepia.ceph.com:9095"}]: dispatch 2024-03-24T11:21:27.026 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:26 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:27.026 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:26 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-24T11:21:27.027 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:26 smithi139 ceph-mon[25719]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-24T11:21:27.027 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:26 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi139.front.sepia.ceph.com:3000"}]: dispatch 2024-03-24T11:21:27.027 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:26 smithi139 ceph-mon[25719]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi139.front.sepia.ceph.com:3000"}]: dispatch 2024-03-24T11:21:27.027 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:26 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:27.027 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:26 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:21:27.027 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:26 smithi139 ceph-mon[25719]: Upgrade: Need to upgrade myself (mgr.x) 2024-03-24T11:21:27.027 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:26 smithi139 ceph-mon[25719]: pgmap v41: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:21:28.948 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:28 smithi062 ceph-mon[29666]: Upgrade: Updating mgr.y 2024-03-24T11:21:28.948 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:28 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:28.948 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:28 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T11:21:28.948 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:28 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T11:21:28.948 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:28 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T11:21:28.948 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:28 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:21:28.949 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:28 smithi062 ceph-mon[29666]: Deploying daemon mgr.y on smithi062 2024-03-24T11:21:28.949 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:28 smithi062 ceph-mon[29666]: pgmap v42: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:21:28.949 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:28 smithi062 ceph-mon[24799]: Upgrade: Updating mgr.y 2024-03-24T11:21:28.949 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:28 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:28.949 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:28 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T11:21:28.949 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:28 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T11:21:28.949 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:28 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T11:21:28.949 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:28 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:21:28.949 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:28 smithi062 ceph-mon[24799]: Deploying daemon mgr.y on smithi062 2024-03-24T11:21:28.949 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:28 smithi062 ceph-mon[24799]: pgmap v42: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:21:29.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:28 smithi139 ceph-mon[25719]: Upgrade: Updating mgr.y 2024-03-24T11:21:29.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:28 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:29.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:28 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T11:21:29.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:28 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T11:21:29.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:28 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T11:21:29.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:28 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:21:29.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:28 smithi139 ceph-mon[25719]: Deploying daemon mgr.y on smithi062 2024-03-24T11:21:29.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:28 smithi139 ceph-mon[25719]: pgmap v42: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:21:29.109 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:28 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[55656]: ts=2024-03-24T11:21:28.658Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"smithi139:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"172.21.15.139:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-24T11:21:29.420 INFO:teuthology.orchestra.run.smithi062.stdout:true 2024-03-24T11:21:29.607 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:29 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:21:29] "GET /metrics HTTP/1.1" 200 38148 "" "Prometheus/2.33.4" 2024-03-24T11:21:29.976 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:29 smithi062 systemd[1]: Stopping Ceph mgr.y for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:21:30.119 INFO:teuthology.orchestra.run.smithi062.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T11:21:30.119 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi062 smithi062 starting - - - - 2024-03-24T11:21:30.119 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi139 smithi139 starting - - - - 2024-03-24T11:21:30.119 INFO:teuthology.orchestra.run.smithi062.stdout:alertmanager.a smithi062 *:9093,9094 starting - - - - 2024-03-24T11:21:30.119 INFO:teuthology.orchestra.run.smithi062.stdout:grafana.a smithi139 *:3000 starting - - - - 2024-03-24T11:21:30.119 INFO:teuthology.orchestra.run.smithi062.stdout:iscsi.foo.smithi062.dggsvk smithi062 running (4m) 90s ago 4m 50.7M - 3.5 e1d6a67b021e dc36997f65c5 2024-03-24T11:21:30.119 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.x smithi139 *:8443,9283 starting - - - - 2024-03-24T11:21:30.119 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.y smithi062 *:9283 running (11m) 90s ago 11m 467M - 17.2.0 e1d6a67b021e 28a8eee27817 2024-03-24T11:21:30.119 INFO:teuthology.orchestra.run.smithi062.stdout:mon.a smithi062 running (11m) 90s ago 11m 50.2M 2048M 17.2.0 e1d6a67b021e 712f01aa9180 2024-03-24T11:21:30.119 INFO:teuthology.orchestra.run.smithi062.stdout:mon.b smithi139 running (10m) 84s ago 10m 43.8M 2048M 17.2.0 e1d6a67b021e 8945cdb4e6aa 2024-03-24T11:21:30.119 INFO:teuthology.orchestra.run.smithi062.stdout:mon.c smithi062 running (10m) 90s ago 10m 40.4M 2048M 17.2.0 e1d6a67b021e 39110a16ab62 2024-03-24T11:21:30.119 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.a smithi062 *:9100 starting - - - - 2024-03-24T11:21:30.120 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.b smithi139 *:9100 starting - - - - 2024-03-24T11:21:30.120 INFO:teuthology.orchestra.run.smithi062.stdout:osd.0 smithi062 running (9m) 90s ago 9m 50.4M 4096M 17.2.0 e1d6a67b021e 8040b30d7170 2024-03-24T11:21:30.120 INFO:teuthology.orchestra.run.smithi062.stdout:osd.1 smithi062 running (9m) 90s ago 9m 52.2M 4096M 17.2.0 e1d6a67b021e e62d22e5dc29 2024-03-24T11:21:30.120 INFO:teuthology.orchestra.run.smithi062.stdout:osd.2 smithi062 running (9m) 90s ago 9m 46.2M 4096M 17.2.0 e1d6a67b021e 714f550103a7 2024-03-24T11:21:30.120 INFO:teuthology.orchestra.run.smithi062.stdout:osd.3 smithi062 running (8m) 90s ago 8m 49.8M 4096M 17.2.0 e1d6a67b021e 1d2868bd5195 2024-03-24T11:21:30.120 INFO:teuthology.orchestra.run.smithi062.stdout:osd.4 smithi139 running (8m) 84s ago 8m 54.5M 4096M 17.2.0 e1d6a67b021e 7568bfa9e74d 2024-03-24T11:21:30.120 INFO:teuthology.orchestra.run.smithi062.stdout:osd.5 smithi139 running (7m) 84s ago 7m 47.7M 4096M 17.2.0 e1d6a67b021e dd0abd599a14 2024-03-24T11:21:30.120 INFO:teuthology.orchestra.run.smithi062.stdout:osd.6 smithi139 running (7m) 84s ago 7m 48.1M 4096M 17.2.0 e1d6a67b021e a4b1289aca69 2024-03-24T11:21:30.120 INFO:teuthology.orchestra.run.smithi062.stdout:osd.7 smithi139 running (7m) 84s ago 7m 50.0M 4096M 17.2.0 e1d6a67b021e bc7fadbc1384 2024-03-24T11:21:30.120 INFO:teuthology.orchestra.run.smithi062.stdout:prometheus.a smithi139 *:9095 starting - - - - 2024-03-24T11:21:30.120 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi062.sacpds smithi062 *:8000 running (4m) 90s ago 4m 69.9M - 17.2.0 e1d6a67b021e 89f4ac308e98 2024-03-24T11:21:30.120 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi139.yljesh smithi139 *:8000 running (4m) 84s ago 4m 69.1M - 17.2.0 e1d6a67b021e e189bdc12cae 2024-03-24T11:21:30.120 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi062.xvpxnk smithi062 *:80 running (4m) 90s ago 4m 69.7M - 17.2.0 e1d6a67b021e d1aa74adab44 2024-03-24T11:21:30.120 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi139.pbtrmp smithi139 *:80 running (4m) 84s ago 4m 72.3M - 17.2.0 e1d6a67b021e 7f8e336755d7 2024-03-24T11:21:30.519 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:21:30.519 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": { 2024-03-24T11:21:30.519 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 3 2024-03-24T11:21:30.519 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:21:30.519 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": { 2024-03-24T11:21:30.519 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 1, 2024-03-24T11:21:30.519 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 1 2024-03-24T11:21:30.519 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:21:30.519 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": { 2024-03-24T11:21:30.520 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 8 2024-03-24T11:21:30.520 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:21:30.520 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": {}, 2024-03-24T11:21:30.520 INFO:teuthology.orchestra.run.smithi062.stdout: "rgw": { 2024-03-24T11:21:30.520 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4 2024-03-24T11:21:30.520 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:21:30.520 INFO:teuthology.orchestra.run.smithi062.stdout: "overall": { 2024-03-24T11:21:30.520 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 16, 2024-03-24T11:21:30.520 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 1 2024-03-24T11:21:30.520 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-24T11:21:30.520 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:21:30.914 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:21:30.914 INFO:teuthology.orchestra.run.smithi062.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69", 2024-03-24T11:21:30.914 INFO:teuthology.orchestra.run.smithi062.stdout: "in_progress": true, 2024-03-24T11:21:30.914 INFO:teuthology.orchestra.run.smithi062.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T11:21:30.914 INFO:teuthology.orchestra.run.smithi062.stdout: "services_complete": [], 2024-03-24T11:21:30.914 INFO:teuthology.orchestra.run.smithi062.stdout: "progress": "", 2024-03-24T11:21:30.914 INFO:teuthology.orchestra.run.smithi062.stdout: "message": "Currently upgrading mgr daemons", 2024-03-24T11:21:30.914 INFO:teuthology.orchestra.run.smithi062.stdout: "is_paused": false 2024-03-24T11:21:30.914 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:21:31.097 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:31 smithi062 ceph-mon[24799]: from='client.25156 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:21:31.097 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:31 smithi062 ceph-mon[24799]: from='client.25213 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:21:31.097 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:31 smithi062 ceph-mon[24799]: from='client.25219 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:21:31.097 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:31 smithi062 ceph-mon[24799]: pgmap v43: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:21:31.097 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:31 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/2731248898' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:21:31.097 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:30 smithi062 podman[66690]: 2024-03-24 11:21:30.817044175 +0000 UTC m=+0.155705604 container died 28a8eee2781708aeffac2cb97b1de29cee6b717a1a4a92db247661393e42e0c9 (image=quay.io/ceph/ceph:v17.2.0, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, RELEASE=HEAD, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=-17.2.0, ceph=True, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, build-date=2022-05-03T08:36:31.336870, io.k8s.display-name=CentOS Stream 8, version=8, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, release=754, vendor=Red Hat, Inc., com.redhat.component=centos-stream-container, io.buildah.version=1.19.8, distribution-scope=public, name=centos-stream, io.openshift.expose-services=, vcs-type=git, io.openshift.tags=base centos centos-stream, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, GIT_REPO=https://github.com/ceph/ceph-container.git, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, maintainer=Guillaume Abrioux , architecture=x86_64) 2024-03-24T11:21:31.097 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:31 smithi062 ceph-mon[29666]: from='client.25156 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:21:31.098 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:31 smithi062 ceph-mon[29666]: from='client.25213 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:21:31.098 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:31 smithi062 ceph-mon[29666]: from='client.25219 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:21:31.098 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:31 smithi062 ceph-mon[29666]: pgmap v43: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:21:31.098 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:31 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/2731248898' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:21:31.301 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:31 smithi139 ceph-mon[25719]: from='client.25156 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:21:31.302 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:31 smithi139 ceph-mon[25719]: from='client.25213 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:21:31.302 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:31 smithi139 ceph-mon[25719]: from='client.25219 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:21:31.302 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:31 smithi139 ceph-mon[25719]: pgmap v43: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:21:31.302 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:31 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/2731248898' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:21:31.321 INFO:teuthology.orchestra.run.smithi062.stdout:HEALTH_OK 2024-03-24T11:21:31.387 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:31 smithi062 podman[66690]: 2024-03-24 11:21:31.186635411 +0000 UTC m=+0.525296849 container cleanup 28a8eee2781708aeffac2cb97b1de29cee6b717a1a4a92db247661393e42e0c9 (image=quay.io/ceph/ceph:v17.2.0, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, maintainer=Guillaume Abrioux , release=754, vendor=Red Hat, Inc., architecture=x86_64, io.openshift.expose-services=, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, com.redhat.component=centos-stream-container, io.k8s.display-name=CentOS Stream 8, distribution-scope=public, GIT_CLEAN=True, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, version=8, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, GIT_REPO=https://github.com/ceph/ceph-container.git, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, build-date=2022-05-03T08:36:31.336870, io.openshift.tags=base centos centos-stream, io.buildah.version=1.19.8, name=centos-stream, vcs-type=git, ceph=True, CEPH_POINT_RELEASE=-17.2.0, GIT_BRANCH=HEAD, RELEASE=HEAD) 2024-03-24T11:21:31.388 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:31 smithi062 bash[66690]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y 2024-03-24T11:21:31.695 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:31 smithi062 podman[66741]: 2024-03-24 11:21:31.388138804 +0000 UTC m=+0.574710558 container remove 28a8eee2781708aeffac2cb97b1de29cee6b717a1a4a92db247661393e42e0c9 (image=quay.io/ceph/ceph:v17.2.0, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y, version=8, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, CEPH_POINT_RELEASE=-17.2.0, RELEASE=HEAD, maintainer=Guillaume Abrioux , url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, GIT_REPO=https://github.com/ceph/ceph-container.git, vendor=Red Hat, Inc., io.openshift.tags=base centos centos-stream, ceph=True, vcs-type=git, GIT_CLEAN=True, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, GIT_BRANCH=HEAD, release=754, distribution-scope=public, io.buildah.version=1.19.8, name=centos-stream, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, io.k8s.display-name=CentOS Stream 8, io.openshift.expose-services=, com.redhat.component=centos-stream-container, build-date=2022-05-03T08:36:31.336870, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, architecture=x86_64) 2024-03-24T11:21:31.695 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:31 smithi062 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-03-24T11:21:31.695 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:31 smithi062 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mgr.y.service: Failed with result 'exit-code'. 2024-03-24T11:21:31.695 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:31 smithi062 systemd[1]: Stopped Ceph mgr.y for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:21:31.695 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:31 smithi062 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mgr.y.service: Consumed 1min 9.154s CPU time. 2024-03-24T11:21:31.976 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:31 smithi062 systemd[1]: Starting Ceph mgr.y for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:21:32.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:32 smithi139 ceph-mon[25719]: from='client.15321 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:21:32.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:32 smithi139 ceph-mon[25719]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:21:32.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:32 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/53848069' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:21:32.267 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:32 smithi062 ceph-mon[24799]: from='client.15321 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:21:32.267 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:32 smithi062 ceph-mon[24799]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:21:32.267 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:32 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/53848069' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:21:32.267 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:32 smithi062 podman[66912]: 2024-03-24 11:21:31.917132743 +0000 UTC m=+0.018082806 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:21:32.267 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:32 smithi062 podman[66912]: 2024-03-24 11:21:32.02310429 +0000 UTC m=+0.124054379 container create 719469f1060d2462606ccb96995751f0291c266d072f633e1ffe150f75c8202c (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, RELEASE=squid-db0330b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T11:21:32.268 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:32 smithi062 ceph-mon[29666]: from='client.15321 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:21:32.268 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:32 smithi062 ceph-mon[29666]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:21:32.268 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:32 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/53848069' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:21:32.587 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:32 smithi062 podman[66912]: 2024-03-24 11:21:32.267455895 +0000 UTC m=+0.368405955 container init 719469f1060d2462606ccb96995751f0291c266d072f633e1ffe150f75c8202c (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=squid-db0330b, ceph=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-24T11:21:32.587 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:32 smithi062 podman[66912]: 2024-03-24 11:21:32.270199684 +0000 UTC m=+0.371149744 container start 719469f1060d2462606ccb96995751f0291c266d072f633e1ffe150f75c8202c (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-24T11:21:32.588 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:32 smithi062 ceph-mgr[66931]: -- 172.21.15.62:0/3277697634 <== mon.0 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56151b0d41a0 con 0x56151b0aac00 2024-03-24T11:21:32.588 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:32 smithi062 bash[66912]: 719469f1060d2462606ccb96995751f0291c266d072f633e1ffe150f75c8202c 2024-03-24T11:21:32.588 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:32 smithi062 systemd[1]: Started Ceph mgr.y for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:21:33.226 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:32 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[66927]: 2024-03-24T11:21:32.849+0000 7f62583c9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-24T11:21:33.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:32 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[66927]: 2024-03-24T11:21:32.950+0000 7f62583c9200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-24T11:21:33.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:33 smithi062 ceph-mon[24799]: pgmap v44: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:21:33.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:33 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:33.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:33 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:33.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:33 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi139", "caps": []}]: dispatch 2024-03-24T11:21:33.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:33 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi062", "caps": []}]: dispatch 2024-03-24T11:21:33.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:33 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi139", "caps": []}]: dispatch 2024-03-24T11:21:33.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:33 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi062", "caps": []}]: dispatch 2024-03-24T11:21:33.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:33 smithi062 ceph-mon[29666]: pgmap v44: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:21:33.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:33 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:33.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:33 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:33.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:33 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi139", "caps": []}]: dispatch 2024-03-24T11:21:33.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:33 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi062", "caps": []}]: dispatch 2024-03-24T11:21:33.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:33 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi139", "caps": []}]: dispatch 2024-03-24T11:21:33.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:33 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi062", "caps": []}]: dispatch 2024-03-24T11:21:33.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:33 smithi139 ceph-mon[25719]: pgmap v44: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:21:33.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:33 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:33.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:33 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:33.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:33 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi139", "caps": []}]: dispatch 2024-03-24T11:21:33.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:33 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi062", "caps": []}]: dispatch 2024-03-24T11:21:33.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:33 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi139", "caps": []}]: dispatch 2024-03-24T11:21:33.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:33 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi062", "caps": []}]: dispatch 2024-03-24T11:21:33.976 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[66927]: 2024-03-24T11:21:33.655+0000 7f62583c9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-24T11:21:33.977 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[66927]: 2024-03-24T11:21:33.755+0000 7f62583c9200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-24T11:21:33.977 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[66927]: 2024-03-24T11:21:33.900+0000 7f62583c9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-24T11:21:34.282 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:34 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[66927]: 2024-03-24T11:21:34.001+0000 7f62583c9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-24T11:21:34.282 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:34 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[66927]: 2024-03-24T11:21:34.088+0000 7f62583c9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-24T11:21:34.583 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:34 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[66927]: 2024-03-24T11:21:34.369+0000 7f62583c9200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-24T11:21:34.584 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:34 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[66927]: 2024-03-24T11:21:34.493+0000 7f62583c9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-24T11:21:34.884 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:34 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[66927]: 2024-03-24T11:21:34.585+0000 7f62583c9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-24T11:21:34.884 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:34 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[66927]: 2024-03-24T11:21:34.692+0000 7f62583c9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-24T11:21:34.884 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:34 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[66927]: 2024-03-24T11:21:34.883+0000 7f62583c9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-24T11:21:35.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:35 smithi139 ceph-mon[25719]: Deploying daemon agent.smithi139 on smithi139 2024-03-24T11:21:35.066 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:35 smithi139 ceph-mon[25719]: Deploying daemon agent.smithi062 on smithi062 2024-03-24T11:21:35.067 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:35 smithi139 ceph-mon[25719]: pgmap v45: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:21:35.067 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:35 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:35.138 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:35 smithi062 ceph-mon[29666]: Deploying daemon agent.smithi139 on smithi139 2024-03-24T11:21:35.139 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:35 smithi062 ceph-mon[29666]: Deploying daemon agent.smithi062 on smithi062 2024-03-24T11:21:35.139 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:35 smithi062 ceph-mon[29666]: pgmap v45: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:21:35.139 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:35 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:35.139 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:35 smithi062 ceph-mon[24799]: Deploying daemon agent.smithi139 on smithi139 2024-03-24T11:21:35.139 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:35 smithi062 ceph-mon[24799]: Deploying daemon agent.smithi062 on smithi062 2024-03-24T11:21:35.139 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:35 smithi062 ceph-mon[24799]: pgmap v45: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:21:35.139 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:35 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:36.210 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:36 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[66927]: 2024-03-24T11:21:36.010+0000 7f62583c9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-24T11:21:36.211 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:36 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[66927]: 2024-03-24T11:21:36.147+0000 7f62583c9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-24T11:21:36.238 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:36 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:36.239 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:36 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:36.239 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:36 smithi139 ceph-mon[25719]: pgmap v46: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:21:36.239 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:36 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:36.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:36 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:36.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:36 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:36.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:36 smithi062 ceph-mon[24799]: pgmap v46: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:21:36.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:36 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:36.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:36 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:36.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:36 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:36.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:36 smithi062 ceph-mon[29666]: pgmap v46: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:21:36.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:36 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:36.808 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:36 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[66927]: 2024-03-24T11:21:36.502+0000 7f62583c9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-24T11:21:36.808 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:36 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[66927]: 2024-03-24T11:21:36.598+0000 7f62583c9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-24T11:21:37.215 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:36 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[66927]: 2024-03-24T11:21:36.807+0000 7f62583c9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-24T11:21:37.215 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:37 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[66927]: 2024-03-24T11:21:37.005+0000 7f62583c9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-24T11:21:37.477 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:37 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[66927]: 2024-03-24T11:21:37.318+0000 7f62583c9200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-24T11:21:37.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:37 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:37.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:37 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:37.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:37 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:37.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:37 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:37.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:37 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:37.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:37 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:37.859 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:37 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[66927]: 2024-03-24T11:21:37.594+0000 7f62583c9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-24T11:21:37.859 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:37 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[66927]: 2024-03-24T11:21:37.858+0000 7f62583c9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-24T11:21:38.937 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:38 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[66927]: 2024-03-24T11:21:38.758+0000 7f62583c9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-24T11:21:38.937 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:38 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[55656]: ts=2024-03-24T11:21:38.659Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"smithi139:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"172.21.15.139:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-24T11:21:39.226 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:38 smithi062 ceph-mon[29666]: pgmap v47: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:21:39.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:38 smithi062 ceph-mon[24799]: pgmap v47: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:21:39.307 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:38 smithi139 ceph-mon[25719]: pgmap v47: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:21:39.307 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:39 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:21:39] "GET /metrics HTTP/1.1" 200 38145 "" "Prometheus/2.33.4" 2024-03-24T11:21:39.727 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:39 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[66927]: 2024-03-24T11:21:39.379+0000 7f62583c9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-24T11:21:39.727 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:39 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[66927]: 2024-03-24T11:21:39.487+0000 7f62583c9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-24T11:21:40.030 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:39 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:21:40.031 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:39 smithi062 ceph-mon[29666]: Standby manager daemon y restarted 2024-03-24T11:21:40.031 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:39 smithi062 ceph-mon[29666]: Standby manager daemon y started 2024-03-24T11:21:40.031 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:39 smithi062 ceph-mon[29666]: from='mgr.? 172.21.15.62:0/992971241' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-24T11:21:40.031 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:39 smithi062 ceph-mon[29666]: from='mgr.? 172.21.15.62:0/992971241' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T11:21:40.031 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:39 smithi062 ceph-mon[29666]: from='mgr.? 172.21.15.62:0/992971241' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-24T11:21:40.031 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:39 smithi062 ceph-mon[29666]: from='mgr.? 172.21.15.62:0/992971241' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T11:21:40.031 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:39 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:21:40.031 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:39 smithi062 ceph-mon[24799]: Standby manager daemon y restarted 2024-03-24T11:21:40.031 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:39 smithi062 ceph-mon[24799]: Standby manager daemon y started 2024-03-24T11:21:40.031 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:39 smithi062 ceph-mon[24799]: from='mgr.? 172.21.15.62:0/992971241' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-24T11:21:40.031 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:39 smithi062 ceph-mon[24799]: from='mgr.? 172.21.15.62:0/992971241' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T11:21:40.032 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:39 smithi062 ceph-mon[24799]: from='mgr.? 172.21.15.62:0/992971241' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-24T11:21:40.032 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:39 smithi062 ceph-mon[24799]: from='mgr.? 172.21.15.62:0/992971241' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T11:21:40.032 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:39 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[66927]: 2024-03-24T11:21:39.770+0000 7f62583c9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-24T11:21:40.032 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:39 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[66927]: [24/Mar/2024:11:21:39] ENGINE Bus STARTING 2024-03-24T11:21:40.032 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:39 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[66927]: CherryPy Checker: 2024-03-24T11:21:40.032 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:39 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[66927]: The Application mounted at '' has an empty config. 2024-03-24T11:21:40.032 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:39 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[66927]: 2024-03-24T11:21:40.032 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:39 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[66927]: [24/Mar/2024:11:21:39] ENGINE Serving on http://:::9283 2024-03-24T11:21:40.032 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:39 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[66927]: [24/Mar/2024:11:21:39] ENGINE Bus STARTED 2024-03-24T11:21:40.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:39 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:21:40.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:39 smithi139 ceph-mon[25719]: Standby manager daemon y restarted 2024-03-24T11:21:40.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:39 smithi139 ceph-mon[25719]: Standby manager daemon y started 2024-03-24T11:21:40.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:39 smithi139 ceph-mon[25719]: from='mgr.? 172.21.15.62:0/992971241' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-24T11:21:40.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:39 smithi139 ceph-mon[25719]: from='mgr.? 172.21.15.62:0/992971241' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T11:21:40.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:39 smithi139 ceph-mon[25719]: from='mgr.? 172.21.15.62:0/992971241' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-24T11:21:40.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:39 smithi139 ceph-mon[25719]: from='mgr.? 172.21.15.62:0/992971241' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T11:21:41.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:40 smithi062 ceph-mon[24799]: from='client.25156 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:21:41.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:40 smithi062 ceph-mon[24799]: pgmap v48: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:21:41.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:40 smithi062 ceph-mon[24799]: mgrmap e27: x(active, since 76s), standbys: y 2024-03-24T11:21:41.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:40 smithi062 ceph-mon[29666]: from='client.25156 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:21:41.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:40 smithi062 ceph-mon[29666]: pgmap v48: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:21:41.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:40 smithi062 ceph-mon[29666]: mgrmap e27: x(active, since 76s), standbys: y 2024-03-24T11:21:41.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:40 smithi139 ceph-mon[25719]: from='client.25156 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:21:41.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:40 smithi139 ceph-mon[25719]: pgmap v48: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:21:41.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:40 smithi139 ceph-mon[25719]: mgrmap e27: x(active, since 76s), standbys: y 2024-03-24T11:21:42.383 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:42 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:42.383 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:42 smithi062 ceph-mon[24799]: pgmap v49: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:21:42.383 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:42 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:42.383 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:42 smithi062 ceph-mon[29666]: pgmap v49: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:21:42.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:42 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:42.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:42 smithi139 ceph-mon[25719]: pgmap v49: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:21:43.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:43 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:43.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:43 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:43.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:43 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:43.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:43 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:43.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:43 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:43.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:43 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:44.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:43 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:44.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:43 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:44.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:43 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:44.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:44 smithi139 ceph-mon[25719]: pgmap v50: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:21:44.976 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:44 smithi062 ceph-mon[24799]: pgmap v50: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:21:44.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:44 smithi062 ceph-mon[29666]: pgmap v50: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:21:46.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:45 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:46.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:45 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:46.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:45 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:46.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:45 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:46.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:45 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:21:46.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:45 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:21:46.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:45 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:46.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:45 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:46.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:45 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi062.dggsvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-03-24T11:21:46.239 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:45 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi062.dggsvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-03-24T11:21:46.239 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:45 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:21:46.239 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:45 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:46.239 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:45 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:46.239 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:45 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:46.239 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:45 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:46.239 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:45 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:21:46.239 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:45 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:21:46.239 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:45 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:46.239 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:45 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:46.239 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:45 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi062.dggsvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-03-24T11:21:46.240 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:45 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi062.dggsvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-03-24T11:21:46.240 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:45 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:21:46.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:45 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:46.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:45 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:46.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:45 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:46.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:45 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:46.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:45 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:21:46.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:45 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:21:46.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:45 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:46.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:45 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:46.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:45 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi062.dggsvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-03-24T11:21:46.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:45 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi062.dggsvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-03-24T11:21:46.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:45 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:21:47.268 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:46 smithi062 ceph-mon[24799]: pgmap v51: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T11:21:47.268 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:46 smithi062 ceph-mon[24799]: Reconfiguring iscsi.foo.smithi062.dggsvk (dependencies changed)... 2024-03-24T11:21:47.268 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:46 smithi062 ceph-mon[24799]: Reconfiguring daemon iscsi.foo.smithi062.dggsvk on smithi062 2024-03-24T11:21:47.268 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:46 smithi062 ceph-mon[24799]: Health check failed: 1 Cephadm Agent(s) are not reporting. Hosts may be offline (CEPHADM_AGENT_DOWN) 2024-03-24T11:21:47.269 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:46 smithi062 ceph-mon[29666]: pgmap v51: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T11:21:47.269 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:46 smithi062 ceph-mon[29666]: Reconfiguring iscsi.foo.smithi062.dggsvk (dependencies changed)... 2024-03-24T11:21:47.269 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:46 smithi062 ceph-mon[29666]: Reconfiguring daemon iscsi.foo.smithi062.dggsvk on smithi062 2024-03-24T11:21:47.269 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:46 smithi062 ceph-mon[29666]: Health check failed: 1 Cephadm Agent(s) are not reporting. Hosts may be offline (CEPHADM_AGENT_DOWN) 2024-03-24T11:21:47.282 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:46 smithi139 ceph-mon[25719]: pgmap v51: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T11:21:47.282 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:46 smithi139 ceph-mon[25719]: Reconfiguring iscsi.foo.smithi062.dggsvk (dependencies changed)... 2024-03-24T11:21:47.282 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:46 smithi139 ceph-mon[25719]: Reconfiguring daemon iscsi.foo.smithi062.dggsvk on smithi062 2024-03-24T11:21:47.282 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:46 smithi139 ceph-mon[25719]: Health check failed: 1 Cephadm Agent(s) are not reporting. Hosts may be offline (CEPHADM_AGENT_DOWN) 2024-03-24T11:21:49.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-mon[25719]: pgmap v52: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 884 B/s rd, 0 op/s 2024-03-24T11:21:49.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:49.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:49.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:49.108 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[55656]: ts=2024-03-24T11:21:48.659Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"smithi139:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"172.21.15.139:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-24T11:21:49.109 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:21:48.888+0000 7fd974920700 -1 log_channel(cephadm) log [ERR] : cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi062.dggsvk ... 2024-03-24T11:21:49.109 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: Creating ceph-iscsi config... 2024-03-24T11:21:49.109 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/iscsi-gateway.cfg 2024-03-24T11:21:49.109 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/tcmu-runner-entrypoint.sh 2024-03-24T11:21:49.109 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: Non-zero exit code 1 from systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk 2024-03-24T11:21:49.109 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: systemctl: stderr Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:21:49.109 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: systemctl: stderr See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:21:49.109 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: Traceback (most recent call last): 2024-03-24T11:21:49.110 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-24T11:21:49.110 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: return _run_code(code, main_globals, None, 2024-03-24T11:21:49.110 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-24T11:21:49.110 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: exec(code, run_globals) 2024-03-24T11:21:49.110 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5520, in 2024-03-24T11:21:49.110 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5508, in main 2024-03-24T11:21:49.110 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3038, in command_deploy_from 2024-03-24T11:21:49.110 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3073, in _common_deploy 2024-03-24T11:21:49.110 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3093, in _deploy_daemon_container 2024-03-24T11:21:49.110 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 1065, in deploy_daemon 2024-03-24T11:21:49.110 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-03-24T11:21:49.110 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: RuntimeError: Failed command: systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk: Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:21:49.110 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:21:49.110 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: Traceback (most recent call last): 2024-03-24T11:21:49.111 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1109, in _check_daemons 2024-03-24T11:21:49.111 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: self.mgr._daemon_action(daemon_spec, action=action) 2024-03-24T11:21:49.111 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/usr/share/ceph/mgr/cephadm/module.py", line 2372, in _daemon_action 2024-03-24T11:21:49.111 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: CephadmServe(self)._create_daemon(daemon_spec, reconfig=(action == 'reconfig'))) 2024-03-24T11:21:49.111 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/usr/share/ceph/mgr/cephadm/module.py", line 739, in wait_async 2024-03-24T11:21:49.111 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: return self.event_loop.get_result(coro, timeout) 2024-03-24T11:21:49.111 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/usr/share/ceph/mgr/cephadm/ssh.py", line 64, in get_result 2024-03-24T11:21:49.111 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: return future.result(timeout) 2024-03-24T11:21:49.111 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/lib64/python3.6/concurrent/futures/_base.py", line 432, in result 2024-03-24T11:21:49.111 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: return self.__get_result() 2024-03-24T11:21:49.111 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/lib64/python3.6/concurrent/futures/_base.py", line 384, in __get_result 2024-03-24T11:21:49.111 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: raise self._exception 2024-03-24T11:21:49.111 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1369, in _create_daemon 2024-03-24T11:21:49.111 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: config_blobs=daemon_spec.final_config, 2024-03-24T11:21:49.111 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1671, in _run_cephadm 2024-03-24T11:21:49.112 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: f'cephadm exited with an error code: {code}, stderr: {err}') 2024-03-24T11:21:49.112 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: orchestrator._interface.OrchestratorError: cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi062.dggsvk ... 2024-03-24T11:21:49.112 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: Creating ceph-iscsi config... 2024-03-24T11:21:49.112 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/iscsi-gateway.cfg 2024-03-24T11:21:49.112 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/tcmu-runner-entrypoint.sh 2024-03-24T11:21:49.112 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: Non-zero exit code 1 from systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk 2024-03-24T11:21:49.112 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: systemctl: stderr Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:21:49.112 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: systemctl: stderr See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:21:49.112 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: Traceback (most recent call last): 2024-03-24T11:21:49.112 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-24T11:21:49.112 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: return _run_code(code, main_globals, None, 2024-03-24T11:21:49.112 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-24T11:21:49.112 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: exec(code, run_globals) 2024-03-24T11:21:49.112 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5520, in 2024-03-24T11:21:49.112 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5508, in main 2024-03-24T11:21:49.113 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3038, in command_deploy_from 2024-03-24T11:21:49.113 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3073, in _common_deploy 2024-03-24T11:21:49.113 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3093, in _deploy_daemon_container 2024-03-24T11:21:49.113 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 1065, in deploy_daemon 2024-03-24T11:21:49.113 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-03-24T11:21:49.113 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: RuntimeError: Failed command: systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk: Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:21:49.113 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:21:49.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:48 smithi062 ceph-mon[24799]: pgmap v52: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 884 B/s rd, 0 op/s 2024-03-24T11:21:49.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:48 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:49.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:48 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:49.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:48 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:49.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:48 smithi062 ceph-mon[29666]: pgmap v52: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 884 B/s rd, 0 op/s 2024-03-24T11:21:49.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:48 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:49.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:48 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:49.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:48 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:49.607 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:49 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:21:49] "GET /metrics HTTP/1.1" 200 38145 "" "Prometheus/2.33.4" 2024-03-24T11:21:50.088 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:49 smithi062 ceph-mon[24799]: cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi062.dggsvk ... 2024-03-24T11:21:50.088 INFO:journalctl@ceph.mon.a.smithi062.stdout: Creating ceph-iscsi config... 2024-03-24T11:21:50.088 INFO:journalctl@ceph.mon.a.smithi062.stdout: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/iscsi-gateway.cfg 2024-03-24T11:21:50.088 INFO:journalctl@ceph.mon.a.smithi062.stdout: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/tcmu-runner-entrypoint.sh 2024-03-24T11:21:50.088 INFO:journalctl@ceph.mon.a.smithi062.stdout: Non-zero exit code 1 from systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk 2024-03-24T11:21:50.088 INFO:journalctl@ceph.mon.a.smithi062.stdout: systemctl: stderr Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:21:50.089 INFO:journalctl@ceph.mon.a.smithi062.stdout: systemctl: stderr See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:21:50.089 INFO:journalctl@ceph.mon.a.smithi062.stdout: Traceback (most recent call last): 2024-03-24T11:21:50.089 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-24T11:21:50.089 INFO:journalctl@ceph.mon.a.smithi062.stdout: return _run_code(code, main_globals, None, 2024-03-24T11:21:50.089 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-24T11:21:50.089 INFO:journalctl@ceph.mon.a.smithi062.stdout: exec(code, run_globals) 2024-03-24T11:21:50.089 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5520, in 2024-03-24T11:21:50.089 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5508, in main 2024-03-24T11:21:50.089 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3038, in command_deploy_from 2024-03-24T11:21:50.089 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3073, in _common_deploy 2024-03-24T11:21:50.089 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3093, in _deploy_daemon_container 2024-03-24T11:21:50.089 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 1065, in deploy_daemon 2024-03-24T11:21:50.089 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-03-24T11:21:50.090 INFO:journalctl@ceph.mon.a.smithi062.stdout: RuntimeError: Failed command: systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk: Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:21:50.090 INFO:journalctl@ceph.mon.a.smithi062.stdout: See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:21:50.090 INFO:journalctl@ceph.mon.a.smithi062.stdout: Traceback (most recent call last): 2024-03-24T11:21:50.090 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1109, in _check_daemons 2024-03-24T11:21:50.090 INFO:journalctl@ceph.mon.a.smithi062.stdout: self.mgr._daemon_action(daemon_spec, action=action) 2024-03-24T11:21:50.090 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/usr/share/ceph/mgr/cephadm/module.py", line 2372, in _daemon_action 2024-03-24T11:21:50.090 INFO:journalctl@ceph.mon.a.smithi062.stdout: CephadmServe(self)._create_daemon(daemon_spec, reconfig=(action == 'reconfig'))) 2024-03-24T11:21:50.090 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/usr/share/ceph/mgr/cephadm/module.py", line 739, in wait_async 2024-03-24T11:21:50.090 INFO:journalctl@ceph.mon.a.smithi062.stdout: return self.event_loop.get_result(coro, timeout) 2024-03-24T11:21:50.090 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/usr/share/ceph/mgr/cephadm/ssh.py", line 64, in get_result 2024-03-24T11:21:50.090 INFO:journalctl@ceph.mon.a.smithi062.stdout: return future.result(timeout) 2024-03-24T11:21:50.090 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib64/python3.6/concurrent/futures/_base.py", line 432, in result 2024-03-24T11:21:50.090 INFO:journalctl@ceph.mon.a.smithi062.stdout: return self.__get_result() 2024-03-24T11:21:50.090 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib64/python3.6/concurrent/futures/_base.py", line 384, in __get_result 2024-03-24T11:21:50.091 INFO:journalctl@ceph.mon.a.smithi062.stdout: raise self._exception 2024-03-24T11:21:50.091 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1369, in _create_daemon 2024-03-24T11:21:50.091 INFO:journalctl@ceph.mon.a.smithi062.stdout: config_blobs=daemon_spec.final_config, 2024-03-24T11:21:50.091 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1671, in _run_cephadm 2024-03-24T11:21:50.091 INFO:journalctl@ceph.mon.a.smithi062.stdout: f'cephadm exited with an error code: {code}, stderr: {err}') 2024-03-24T11:21:50.091 INFO:journalctl@ceph.mon.a.smithi062.stdout: orchestrator._interface.OrchestratorError: cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi062.dggsvk ... 2024-03-24T11:21:50.091 INFO:journalctl@ceph.mon.a.smithi062.stdout: Creating ceph-iscsi config... 2024-03-24T11:21:50.091 INFO:journalctl@ceph.mon.a.smithi062.stdout: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/iscsi-gateway.cfg 2024-03-24T11:21:50.091 INFO:journalctl@ceph.mon.a.smithi062.stdout: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/tcmu-runner-entrypoint.sh 2024-03-24T11:21:50.091 INFO:journalctl@ceph.mon.a.smithi062.stdout: Non-zero exit code 1 from systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk 2024-03-24T11:21:50.091 INFO:journalctl@ceph.mon.a.smithi062.stdout: systemctl: stderr Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:21:50.091 INFO:journalctl@ceph.mon.a.smithi062.stdout: systemctl: stderr See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:21:50.091 INFO:journalctl@ceph.mon.a.smithi062.stdout: Traceback (most recent call last): 2024-03-24T11:21:50.092 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-24T11:21:50.092 INFO:journalctl@ceph.mon.a.smithi062.stdout: return _run_code(code, main_globals, None, 2024-03-24T11:21:50.092 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-24T11:21:50.092 INFO:journalctl@ceph.mon.a.smithi062.stdout: exec(code, run_globals) 2024-03-24T11:21:50.092 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5520, in 2024-03-24T11:21:50.092 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5508, in main 2024-03-24T11:21:50.092 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3038, in command_deploy_from 2024-03-24T11:21:50.092 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3073, in _common_deploy 2024-03-24T11:21:50.092 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:49 smithi062 ceph-mon[29666]: cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi062.dggsvk ... 2024-03-24T11:21:50.092 INFO:journalctl@ceph.mon.c.smithi062.stdout: Creating ceph-iscsi config... 2024-03-24T11:21:50.093 INFO:journalctl@ceph.mon.c.smithi062.stdout: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/iscsi-gateway.cfg 2024-03-24T11:21:50.093 INFO:journalctl@ceph.mon.c.smithi062.stdout: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/tcmu-runner-entrypoint.sh 2024-03-24T11:21:50.093 INFO:journalctl@ceph.mon.c.smithi062.stdout: Non-zero exit code 1 from systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk 2024-03-24T11:21:50.093 INFO:journalctl@ceph.mon.c.smithi062.stdout: systemctl: stderr Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:21:50.093 INFO:journalctl@ceph.mon.c.smithi062.stdout: systemctl: stderr See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:21:50.093 INFO:journalctl@ceph.mon.c.smithi062.stdout: Traceback (most recent call last): 2024-03-24T11:21:50.093 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-24T11:21:50.093 INFO:journalctl@ceph.mon.c.smithi062.stdout: return _run_code(code, main_globals, None, 2024-03-24T11:21:50.093 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-24T11:21:50.093 INFO:journalctl@ceph.mon.c.smithi062.stdout: exec(code, run_globals) 2024-03-24T11:21:50.093 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5520, in 2024-03-24T11:21:50.093 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5508, in main 2024-03-24T11:21:50.093 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3038, in command_deploy_from 2024-03-24T11:21:50.094 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3073, in _common_deploy 2024-03-24T11:21:50.094 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3093, in _deploy_daemon_container 2024-03-24T11:21:50.094 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 1065, in deploy_daemon 2024-03-24T11:21:50.094 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-03-24T11:21:50.094 INFO:journalctl@ceph.mon.c.smithi062.stdout: RuntimeError: Failed command: systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk: Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:21:50.094 INFO:journalctl@ceph.mon.c.smithi062.stdout: See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:21:50.094 INFO:journalctl@ceph.mon.c.smithi062.stdout: Traceback (most recent call last): 2024-03-24T11:21:50.094 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1109, in _check_daemons 2024-03-24T11:21:50.094 INFO:journalctl@ceph.mon.c.smithi062.stdout: self.mgr._daemon_action(daemon_spec, action=action) 2024-03-24T11:21:50.094 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/usr/share/ceph/mgr/cephadm/module.py", line 2372, in _daemon_action 2024-03-24T11:21:50.094 INFO:journalctl@ceph.mon.c.smithi062.stdout: CephadmServe(self)._create_daemon(daemon_spec, reconfig=(action == 'reconfig'))) 2024-03-24T11:21:50.094 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/usr/share/ceph/mgr/cephadm/module.py", line 739, in wait_async 2024-03-24T11:21:50.094 INFO:journalctl@ceph.mon.c.smithi062.stdout: return self.event_loop.get_result(coro, timeout) 2024-03-24T11:21:50.095 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/usr/share/ceph/mgr/cephadm/ssh.py", line 64, in get_result 2024-03-24T11:21:50.095 INFO:journalctl@ceph.mon.c.smithi062.stdout: return future.result(timeout) 2024-03-24T11:21:50.095 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/lib64/python3.6/concurrent/futures/_base.py", line 432, in result 2024-03-24T11:21:50.095 INFO:journalctl@ceph.mon.c.smithi062.stdout: return self.__get_result() 2024-03-24T11:21:50.095 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/lib64/python3.6/concurrent/futures/_base.py", line 384, in __get_result 2024-03-24T11:21:50.095 INFO:journalctl@ceph.mon.c.smithi062.stdout: raise self._exception 2024-03-24T11:21:50.095 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1369, in _create_daemon 2024-03-24T11:21:50.095 INFO:journalctl@ceph.mon.c.smithi062.stdout: config_blobs=daemon_spec.final_config, 2024-03-24T11:21:50.095 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1671, in _run_cephadm 2024-03-24T11:21:50.095 INFO:journalctl@ceph.mon.c.smithi062.stdout: f'cephadm exited with an error code: {code}, stderr: {err}') 2024-03-24T11:21:50.095 INFO:journalctl@ceph.mon.c.smithi062.stdout: orchestrator._interface.OrchestratorError: cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi062.dggsvk ... 2024-03-24T11:21:50.095 INFO:journalctl@ceph.mon.c.smithi062.stdout: Creating ceph-iscsi config... 2024-03-24T11:21:50.095 INFO:journalctl@ceph.mon.c.smithi062.stdout: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/iscsi-gateway.cfg 2024-03-24T11:21:50.096 INFO:journalctl@ceph.mon.c.smithi062.stdout: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/tcmu-runner-entrypoint.sh 2024-03-24T11:21:50.096 INFO:journalctl@ceph.mon.c.smithi062.stdout: Non-zero exit code 1 from systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk 2024-03-24T11:21:50.096 INFO:journalctl@ceph.mon.c.smithi062.stdout: systemctl: stderr Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:21:50.096 INFO:journalctl@ceph.mon.c.smithi062.stdout: systemctl: stderr See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:21:50.096 INFO:journalctl@ceph.mon.c.smithi062.stdout: Traceback (most recent call last): 2024-03-24T11:21:50.096 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-24T11:21:50.096 INFO:journalctl@ceph.mon.c.smithi062.stdout: return _run_code(code, main_globals, None, 2024-03-24T11:21:50.096 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-24T11:21:50.096 INFO:journalctl@ceph.mon.c.smithi062.stdout: exec(code, run_globals) 2024-03-24T11:21:50.096 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5520, in 2024-03-24T11:21:50.096 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5508, in main 2024-03-24T11:21:50.096 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3038, in command_deploy_from 2024-03-24T11:21:50.096 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3073, in _common_deploy 2024-03-24T11:21:50.097 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3093, in _deploy_daemon_container 2024-03-24T11:21:50.097 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 1065, in deploy_daemon 2024-03-24T11:21:50.097 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-03-24T11:21:50.097 INFO:journalctl@ceph.mon.a.smithi062.stdout: RuntimeError: Failed command: systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk: Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:21:50.097 INFO:journalctl@ceph.mon.a.smithi062.stdout: See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:21:50.097 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:49 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:21:50.097 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:49 smithi062 ceph-mon[24799]: Upgrade: Need to upgrade myself (mgr.x) 2024-03-24T11:21:50.097 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:49 smithi062 ceph-mon[24799]: Upgrade: Need to upgrade myself (mgr.x) 2024-03-24T11:21:50.097 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3093, in _deploy_daemon_container 2024-03-24T11:21:50.098 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 1065, in deploy_daemon 2024-03-24T11:21:50.098 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-03-24T11:21:50.098 INFO:journalctl@ceph.mon.c.smithi062.stdout: RuntimeError: Failed command: systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk: Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:21:50.098 INFO:journalctl@ceph.mon.c.smithi062.stdout: See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:21:50.098 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:49 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:21:50.098 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:49 smithi062 ceph-mon[29666]: Upgrade: Need to upgrade myself (mgr.x) 2024-03-24T11:21:50.098 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:49 smithi062 ceph-mon[29666]: Upgrade: Need to upgrade myself (mgr.x) 2024-03-24T11:21:50.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:49 smithi139 ceph-mon[25719]: cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi062.dggsvk ... 2024-03-24T11:21:50.358 INFO:journalctl@ceph.mon.b.smithi139.stdout: Creating ceph-iscsi config... 2024-03-24T11:21:50.358 INFO:journalctl@ceph.mon.b.smithi139.stdout: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/iscsi-gateway.cfg 2024-03-24T11:21:50.358 INFO:journalctl@ceph.mon.b.smithi139.stdout: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/tcmu-runner-entrypoint.sh 2024-03-24T11:21:50.358 INFO:journalctl@ceph.mon.b.smithi139.stdout: Non-zero exit code 1 from systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk 2024-03-24T11:21:50.358 INFO:journalctl@ceph.mon.b.smithi139.stdout: systemctl: stderr Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:21:50.358 INFO:journalctl@ceph.mon.b.smithi139.stdout: systemctl: stderr See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:21:50.359 INFO:journalctl@ceph.mon.b.smithi139.stdout: Traceback (most recent call last): 2024-03-24T11:21:50.359 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-24T11:21:50.359 INFO:journalctl@ceph.mon.b.smithi139.stdout: return _run_code(code, main_globals, None, 2024-03-24T11:21:50.359 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-24T11:21:50.359 INFO:journalctl@ceph.mon.b.smithi139.stdout: exec(code, run_globals) 2024-03-24T11:21:50.359 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5520, in 2024-03-24T11:21:50.359 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5508, in main 2024-03-24T11:21:50.359 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3038, in command_deploy_from 2024-03-24T11:21:50.359 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3073, in _common_deploy 2024-03-24T11:21:50.359 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3093, in _deploy_daemon_container 2024-03-24T11:21:50.359 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 1065, in deploy_daemon 2024-03-24T11:21:50.359 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-03-24T11:21:50.359 INFO:journalctl@ceph.mon.b.smithi139.stdout: RuntimeError: Failed command: systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk: Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:21:50.359 INFO:journalctl@ceph.mon.b.smithi139.stdout: See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:21:50.360 INFO:journalctl@ceph.mon.b.smithi139.stdout: Traceback (most recent call last): 2024-03-24T11:21:50.360 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1109, in _check_daemons 2024-03-24T11:21:50.360 INFO:journalctl@ceph.mon.b.smithi139.stdout: self.mgr._daemon_action(daemon_spec, action=action) 2024-03-24T11:21:50.360 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/usr/share/ceph/mgr/cephadm/module.py", line 2372, in _daemon_action 2024-03-24T11:21:50.360 INFO:journalctl@ceph.mon.b.smithi139.stdout: CephadmServe(self)._create_daemon(daemon_spec, reconfig=(action == 'reconfig'))) 2024-03-24T11:21:50.360 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/usr/share/ceph/mgr/cephadm/module.py", line 739, in wait_async 2024-03-24T11:21:50.360 INFO:journalctl@ceph.mon.b.smithi139.stdout: return self.event_loop.get_result(coro, timeout) 2024-03-24T11:21:50.360 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/usr/share/ceph/mgr/cephadm/ssh.py", line 64, in get_result 2024-03-24T11:21:50.360 INFO:journalctl@ceph.mon.b.smithi139.stdout: return future.result(timeout) 2024-03-24T11:21:50.360 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/lib64/python3.6/concurrent/futures/_base.py", line 432, in result 2024-03-24T11:21:50.360 INFO:journalctl@ceph.mon.b.smithi139.stdout: return self.__get_result() 2024-03-24T11:21:50.360 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/lib64/python3.6/concurrent/futures/_base.py", line 384, in __get_result 2024-03-24T11:21:50.360 INFO:journalctl@ceph.mon.b.smithi139.stdout: raise self._exception 2024-03-24T11:21:50.360 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1369, in _create_daemon 2024-03-24T11:21:50.360 INFO:journalctl@ceph.mon.b.smithi139.stdout: config_blobs=daemon_spec.final_config, 2024-03-24T11:21:50.361 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1671, in _run_cephadm 2024-03-24T11:21:50.361 INFO:journalctl@ceph.mon.b.smithi139.stdout: f'cephadm exited with an error code: {code}, stderr: {err}') 2024-03-24T11:21:50.361 INFO:journalctl@ceph.mon.b.smithi139.stdout: orchestrator._interface.OrchestratorError: cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi062.dggsvk ... 2024-03-24T11:21:50.361 INFO:journalctl@ceph.mon.b.smithi139.stdout: Creating ceph-iscsi config... 2024-03-24T11:21:50.361 INFO:journalctl@ceph.mon.b.smithi139.stdout: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/iscsi-gateway.cfg 2024-03-24T11:21:50.361 INFO:journalctl@ceph.mon.b.smithi139.stdout: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/tcmu-runner-entrypoint.sh 2024-03-24T11:21:50.361 INFO:journalctl@ceph.mon.b.smithi139.stdout: Non-zero exit code 1 from systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk 2024-03-24T11:21:50.361 INFO:journalctl@ceph.mon.b.smithi139.stdout: systemctl: stderr Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:21:50.361 INFO:journalctl@ceph.mon.b.smithi139.stdout: systemctl: stderr See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:21:50.361 INFO:journalctl@ceph.mon.b.smithi139.stdout: Traceback (most recent call last): 2024-03-24T11:21:50.361 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-24T11:21:50.361 INFO:journalctl@ceph.mon.b.smithi139.stdout: return _run_code(code, main_globals, None, 2024-03-24T11:21:50.361 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-24T11:21:50.361 INFO:journalctl@ceph.mon.b.smithi139.stdout: exec(code, run_globals) 2024-03-24T11:21:50.362 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5520, in 2024-03-24T11:21:50.362 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5508, in main 2024-03-24T11:21:50.362 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3038, in command_deploy_from 2024-03-24T11:21:50.362 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3073, in _common_deploy 2024-03-24T11:21:50.362 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3093, in _deploy_daemon_container 2024-03-24T11:21:50.362 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 1065, in deploy_daemon 2024-03-24T11:21:50.362 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-03-24T11:21:50.362 INFO:journalctl@ceph.mon.b.smithi139.stdout: RuntimeError: Failed command: systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk: Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:21:50.362 INFO:journalctl@ceph.mon.b.smithi139.stdout: See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:21:50.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:49 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:21:50.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:49 smithi139 ceph-mon[25719]: Upgrade: Need to upgrade myself (mgr.x) 2024-03-24T11:21:50.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:49 smithi139 ceph-mon[25719]: Upgrade: Need to upgrade myself (mgr.x) 2024-03-24T11:21:51.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:50 smithi062 ceph-mon[24799]: pgmap v53: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 884 B/s rd, 0 op/s 2024-03-24T11:21:51.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:50 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T11:21:51.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:50 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T11:21:51.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:50 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T11:21:51.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:50 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:21:51.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:50 smithi062 ceph-mon[29666]: pgmap v53: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 884 B/s rd, 0 op/s 2024-03-24T11:21:51.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:50 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T11:21:51.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:50 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T11:21:51.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:50 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T11:21:51.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:50 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:21:51.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:50 smithi139 ceph-mon[25719]: pgmap v53: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 884 B/s rd, 0 op/s 2024-03-24T11:21:51.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:50 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T11:21:51.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:50 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T11:21:51.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:50 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T11:21:51.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:50 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:21:52.205 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:51 smithi062 ceph-mon[24799]: Upgrade: Updating mgr.y 2024-03-24T11:21:52.205 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:51 smithi062 ceph-mon[24799]: Deploying daemon mgr.y on smithi062 2024-03-24T11:21:52.205 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:51 smithi062 ceph-mon[29666]: Upgrade: Updating mgr.y 2024-03-24T11:21:52.205 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:51 smithi062 ceph-mon[29666]: Deploying daemon mgr.y on smithi062 2024-03-24T11:21:52.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:51 smithi139 ceph-mon[25719]: Upgrade: Updating mgr.y 2024-03-24T11:21:52.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:51 smithi139 ceph-mon[25719]: Deploying daemon mgr.y on smithi062 2024-03-24T11:21:52.477 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:52 smithi062 systemd[1]: Stopping Ceph mgr.y for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:21:52.894 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:52 smithi062 podman[69275]: 2024-03-24 11:21:52.615820745 +0000 UTC m=+0.163383988 container died 719469f1060d2462606ccb96995751f0291c266d072f633e1ffe150f75c8202c (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, RELEASE=squid-db0330b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-24T11:21:53.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:52 smithi062 ceph-mon[24799]: pgmap v54: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 707 B/s rd, 0 op/s 2024-03-24T11:21:53.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:52 smithi062 ceph-mon[29666]: pgmap v54: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 707 B/s rd, 0 op/s 2024-03-24T11:21:53.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:53 smithi062 podman[69275]: 2024-03-24 11:21:53.102002165 +0000 UTC m=+0.649565453 container cleanup 719469f1060d2462606ccb96995751f0291c266d072f633e1ffe150f75c8202c (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T11:21:53.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:53 smithi062 bash[69275]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y 2024-03-24T11:21:53.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:52 smithi139 ceph-mon[25719]: pgmap v54: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 707 B/s rd, 0 op/s 2024-03-24T11:21:53.726 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:53 smithi062 podman[69290]: 2024-03-24 11:21:53.420103264 +0000 UTC m=+0.803505385 container remove 719469f1060d2462606ccb96995751f0291c266d072f633e1ffe150f75c8202c (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T11:21:53.727 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:53 smithi062 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-03-24T11:21:53.727 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:53 smithi062 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mgr.y.service: Failed with result 'exit-code'. 2024-03-24T11:21:53.727 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:53 smithi062 systemd[1]: Stopped Ceph mgr.y for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:21:53.727 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:53 smithi062 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mgr.y.service: Consumed 9.129s CPU time. 2024-03-24T11:21:54.086 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:53 smithi062 systemd[1]: Starting Ceph mgr.y for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:21:54.086 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:53 smithi062 podman[69423]: 2024-03-24 11:21:53.96278906 +0000 UTC m=+0.099185712 container create ca1a49028d652cca6e2c7762d3a815390dfe41049205724bca556a6566cec60b (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-24T11:21:54.086 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:53 smithi062 podman[69423]: 2024-03-24 11:21:53.881999042 +0000 UTC m=+0.018395694 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:21:54.476 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:54 smithi062 podman[69423]: 2024-03-24 11:21:54.190909056 +0000 UTC m=+0.327305708 container init ca1a49028d652cca6e2c7762d3a815390dfe41049205724bca556a6566cec60b (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, RELEASE=squid-db0330b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True) 2024-03-24T11:21:54.477 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:54 smithi062 podman[69423]: 2024-03-24 11:21:54.193446146 +0000 UTC m=+0.329842803 container start ca1a49028d652cca6e2c7762d3a815390dfe41049205724bca556a6566cec60b (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-db0330b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307) 2024-03-24T11:21:54.477 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:54 smithi062 ceph-mgr[69442]: -- 172.21.15.62:0/2933852763 <== mon.0 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5585d3c321a0 con 0x5585d3c08c00 2024-03-24T11:21:54.477 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:54 smithi062 bash[69423]: ca1a49028d652cca6e2c7762d3a815390dfe41049205724bca556a6566cec60b 2024-03-24T11:21:54.477 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:54 smithi062 systemd[1]: Started Ceph mgr.y for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:21:55.051 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:54 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:21:54.765+0000 7f95ff594200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-24T11:21:55.051 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:54 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:21:54.864+0000 7f95ff594200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-24T11:21:55.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:55 smithi139 ceph-mon[25719]: pgmap v55: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 707 B/s rd, 0 op/s 2024-03-24T11:21:55.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:55 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:55.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:55 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:21:55.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:55 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:55.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:55 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:55.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:55 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:21:55.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:55 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:21:55.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:55 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:55.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:55 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi062.dggsvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-03-24T11:21:55.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:55 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi062.dggsvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-03-24T11:21:55.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:55 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:21:55.386 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:55 smithi062 ceph-mon[24799]: pgmap v55: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 707 B/s rd, 0 op/s 2024-03-24T11:21:55.387 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:55 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:55.387 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:55 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:21:55.387 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:55 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:55.387 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:55 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:55.387 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:55 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:21:55.387 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:55 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:21:55.387 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:55 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:55.387 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:55 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi062.dggsvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-03-24T11:21:55.387 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:55 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi062.dggsvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-03-24T11:21:55.387 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:55 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:21:55.387 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:55 smithi062 ceph-mon[29666]: pgmap v55: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 707 B/s rd, 0 op/s 2024-03-24T11:21:55.388 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:55 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:55.388 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:55 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:21:55.388 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:55 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:55.388 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:55 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:55.388 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:55 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:21:55.388 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:55 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:21:55.388 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:55 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:55.388 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:55 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi062.dggsvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-03-24T11:21:55.388 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:55 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi062.dggsvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-03-24T11:21:55.388 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:55 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:21:55.669 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:55 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:21:55.568+0000 7f95ff594200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-24T11:21:55.932 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:55 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:21:55.669+0000 7f95ff594200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-24T11:21:55.932 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:55 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:21:55.830+0000 7f95ff594200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-24T11:21:56.195 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:55 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:21:55.931+0000 7f95ff594200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-24T11:21:56.195 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:56 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:21:56.022+0000 7f95ff594200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-24T11:21:56.195 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:56 smithi062 ceph-mon[24799]: pgmap v56: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 784 B/s rd, 0 op/s 2024-03-24T11:21:56.195 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:56 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:21:56.195 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:56 smithi062 ceph-mon[24799]: Reconfiguring iscsi.foo.smithi062.dggsvk (dependencies changed)... 2024-03-24T11:21:56.196 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:56 smithi062 ceph-mon[24799]: Reconfiguring daemon iscsi.foo.smithi062.dggsvk on smithi062 2024-03-24T11:21:56.196 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:56 smithi062 ceph-mon[24799]: Health check cleared: CEPHADM_AGENT_DOWN (was: 1 Cephadm Agent(s) are not reporting. Hosts may be offline) 2024-03-24T11:21:56.196 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:56 smithi062 ceph-mon[24799]: Cluster is now healthy 2024-03-24T11:21:56.196 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:56 smithi062 ceph-mon[29666]: pgmap v56: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 784 B/s rd, 0 op/s 2024-03-24T11:21:56.196 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:56 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:21:56.196 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:56 smithi062 ceph-mon[29666]: Reconfiguring iscsi.foo.smithi062.dggsvk (dependencies changed)... 2024-03-24T11:21:56.196 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:56 smithi062 ceph-mon[29666]: Reconfiguring daemon iscsi.foo.smithi062.dggsvk on smithi062 2024-03-24T11:21:56.196 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:56 smithi062 ceph-mon[29666]: Health check cleared: CEPHADM_AGENT_DOWN (was: 1 Cephadm Agent(s) are not reporting. Hosts may be offline) 2024-03-24T11:21:56.196 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:56 smithi062 ceph-mon[29666]: Cluster is now healthy 2024-03-24T11:21:56.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:56 smithi139 ceph-mon[25719]: pgmap v56: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 784 B/s rd, 0 op/s 2024-03-24T11:21:56.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:56 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:21:56.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:56 smithi139 ceph-mon[25719]: Reconfiguring iscsi.foo.smithi062.dggsvk (dependencies changed)... 2024-03-24T11:21:56.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:56 smithi139 ceph-mon[25719]: Reconfiguring daemon iscsi.foo.smithi062.dggsvk on smithi062 2024-03-24T11:21:56.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:56 smithi139 ceph-mon[25719]: Health check cleared: CEPHADM_AGENT_DOWN (was: 1 Cephadm Agent(s) are not reporting. Hosts may be offline) 2024-03-24T11:21:56.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:56 smithi139 ceph-mon[25719]: Cluster is now healthy 2024-03-24T11:21:56.475 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:56 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:21:56.304+0000 7f95ff594200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-24T11:21:56.476 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:56 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:21:56.437+0000 7f95ff594200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-24T11:21:56.726 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:56 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:21:56.539+0000 7f95ff594200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-24T11:21:56.727 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:56 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:21:56.644+0000 7f95ff594200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-24T11:21:57.207 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:56 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:21:56.826+0000 7f95ff594200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-24T11:21:58.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:58 smithi062 ceph-mon[24799]: pgmap v57: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 282 B/s rd, 0 op/s 2024-03-24T11:21:58.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:58 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:58.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:58 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:58.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:21:58 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:58.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:57 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:21:57.867+0000 7f95ff594200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-24T11:21:58.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:58 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:21:58.010+0000 7f95ff594200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-24T11:21:58.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:58 smithi062 ceph-mon[29666]: pgmap v57: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 282 B/s rd, 0 op/s 2024-03-24T11:21:58.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:58 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:58.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:58 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:58.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:21:58 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:58.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:58 smithi139 ceph-mon[25719]: pgmap v57: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 282 B/s rd, 0 op/s 2024-03-24T11:21:58.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:58 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:58.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:58 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:58.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:21:58 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:21:58.673 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:58 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:21:58.362+0000 7f95ff594200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-24T11:21:58.673 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:58 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:21:58.467+0000 7f95ff594200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-24T11:21:58.976 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:58 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:21:58.672+0000 7f95ff594200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-24T11:21:58.976 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:58 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:21:58.872+0000 7f95ff594200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-24T11:21:59.057 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:21:58 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[55656]: ts=2024-03-24T11:21:58.659Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"smithi139:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"172.21.15.139:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-24T11:21:59.357 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:21:59 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:21:59] "GET /metrics HTTP/1.1" 200 38220 "" "Prometheus/2.33.4" 2024-03-24T11:21:59.429 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:59 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:21:59.185+0000 7f95ff594200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-24T11:21:59.726 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:59 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:21:59.473+0000 7f95ff594200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-24T11:22:00.069 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:21:59 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:21:59.734+0000 7f95ff594200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-24T11:22:00.336 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:00 smithi062 ceph-mon[29666]: pgmap v58: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 282 B/s rd, 0 op/s 2024-03-24T11:22:00.336 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:00 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:22:00.336 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:00 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:22:00.336 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:00 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-03-24T11:22:00.336 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:00 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-03-24T11:22:00.336 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:00 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:00.337 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:00 smithi062 ceph-mon[29666]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-03-24T11:22:00.337 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:00 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-03-24T11:22:00.337 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:00 smithi062 ceph-mon[29666]: osdmap e106: 8 total, 8 up, 8 in 2024-03-24T11:22:00.337 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:00 smithi062 ceph-mon[29666]: Standby manager daemon x started 2024-03-24T11:22:00.337 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:00 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/4288352954' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-24T11:22:00.337 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:00 smithi062 ceph-mon[24799]: pgmap v58: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 282 B/s rd, 0 op/s 2024-03-24T11:22:00.337 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:00 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:22:00.337 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:00 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:22:00.337 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:00 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-03-24T11:22:00.337 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:00 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-03-24T11:22:00.337 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:00 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:00.337 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:00 smithi062 ceph-mon[24799]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-03-24T11:22:00.338 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:00 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-03-24T11:22:00.338 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:00 smithi062 ceph-mon[24799]: osdmap e106: 8 total, 8 up, 8 in 2024-03-24T11:22:00.338 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:00 smithi062 ceph-mon[24799]: Standby manager daemon x started 2024-03-24T11:22:00.338 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:00 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/4288352954' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-24T11:22:00.338 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:00 smithi139 ceph-mon[25719]: pgmap v58: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 282 B/s rd, 0 op/s 2024-03-24T11:22:00.338 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:00 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:22:00.338 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:00 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' 2024-03-24T11:22:00.338 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:00 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-03-24T11:22:00.339 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:00 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-03-24T11:22:00.339 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:00 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:00.339 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:00 smithi139 ceph-mon[25719]: from='mgr.24893 172.21.15.139:0/1098719313' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-03-24T11:22:00.339 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:00 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-03-24T11:22:00.339 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:00 smithi139 ceph-mon[25719]: osdmap e106: 8 total, 8 up, 8 in 2024-03-24T11:22:00.339 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:00 smithi139 ceph-mon[25719]: Standby manager daemon x started 2024-03-24T11:22:00.339 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:00 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/4288352954' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-24T11:22:00.339 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:00 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:22:00.285+0000 7fd9e2269700 -1 mgr handle_mgr_map I was active but no longer am 2024-03-24T11:22:00.608 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:00 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: ignoring --setuser ceph since I am not root 2024-03-24T11:22:00.608 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:00 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: ignoring --setgroup ceph since I am not root 2024-03-24T11:22:00.608 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:00 smithi139 ceph-mgr[52321]: -- 172.21.15.139:0/2710453078 <== mon.2 v2:172.21.15.139:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f2d7df61a0 con 0x55f2d7dccc00 2024-03-24T11:22:00.976 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:00 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:22:00.661+0000 7f95ff594200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-24T11:22:01.108 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:00 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:22:00.844+0000 7fcccb69c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-24T11:22:01.108 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:00 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:22:00.938+0000 7fcccb69c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-24T11:22:01.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:01 smithi139 ceph-mon[25719]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-03-24T11:22:01.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:01 smithi139 ceph-mon[25719]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-03-24T11:22:01.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:01 smithi139 ceph-mon[25719]: Upgrade: Need to upgrade myself (mgr.x) 2024-03-24T11:22:01.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:01 smithi139 ceph-mon[25719]: Upgrade: Need to upgrade myself (mgr.x) 2024-03-24T11:22:01.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:01 smithi139 ceph-mon[25719]: Failing over to other MGR 2024-03-24T11:22:01.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:01 smithi139 ceph-mon[25719]: from='mgr.24893 ' entity='mgr.x' cmd='[{"prefix": "mgr fail", "who": "x"}]': finished 2024-03-24T11:22:01.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:01 smithi139 ceph-mon[25719]: mgrmap e28: y(active, starting, since 0.954294s), standbys: x 2024-03-24T11:22:01.282 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:01 smithi062 ceph-mon[29666]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-03-24T11:22:01.282 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:01 smithi062 ceph-mon[29666]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-03-24T11:22:01.282 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:01 smithi062 ceph-mon[29666]: Upgrade: Need to upgrade myself (mgr.x) 2024-03-24T11:22:01.282 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:01 smithi062 ceph-mon[29666]: Upgrade: Need to upgrade myself (mgr.x) 2024-03-24T11:22:01.282 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:01 smithi062 ceph-mon[29666]: Failing over to other MGR 2024-03-24T11:22:01.282 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:01 smithi062 ceph-mon[29666]: from='mgr.24893 ' entity='mgr.x' cmd='[{"prefix": "mgr fail", "who": "x"}]': finished 2024-03-24T11:22:01.282 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:01 smithi062 ceph-mon[29666]: mgrmap e28: y(active, starting, since 0.954294s), standbys: x 2024-03-24T11:22:01.283 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:01 smithi062 ceph-mon[24799]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-03-24T11:22:01.283 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:01 smithi062 ceph-mon[24799]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-03-24T11:22:01.283 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:01 smithi062 ceph-mon[24799]: Upgrade: Need to upgrade myself (mgr.x) 2024-03-24T11:22:01.283 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:01 smithi062 ceph-mon[24799]: Upgrade: Need to upgrade myself (mgr.x) 2024-03-24T11:22:01.283 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:01 smithi062 ceph-mon[24799]: Failing over to other MGR 2024-03-24T11:22:01.283 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:01 smithi062 ceph-mon[24799]: from='mgr.24893 ' entity='mgr.x' cmd='[{"prefix": "mgr fail", "who": "x"}]': finished 2024-03-24T11:22:01.283 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:01 smithi062 ceph-mon[24799]: mgrmap e28: y(active, starting, since 0.954294s), standbys: x 2024-03-24T11:22:01.622 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:01 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:22:01.281+0000 7f95ff594200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-24T11:22:01.623 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:01 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:22:01.389+0000 7f95ff594200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-24T11:22:01.623 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:01 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:22:01.621+0000 7f95ff594200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-24T11:22:01.858 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:01 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:22:01.578+0000 7fcccb69c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-24T11:22:01.858 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:01 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:22:01.673+0000 7fcccb69c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-24T11:22:01.858 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:01 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:22:01.815+0000 7fcccb69c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-24T11:22:01.976 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:01 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: [24/Mar/2024:11:22:01] ENGINE Bus STARTING 2024-03-24T11:22:01.977 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:01 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: CherryPy Checker: 2024-03-24T11:22:01.977 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:01 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: The Application mounted at '' has an empty config. 2024-03-24T11:22:01.977 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:01 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:22:01.977 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:01 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: [24/Mar/2024:11:22:01] ENGINE Serving on http://:::9283 2024-03-24T11:22:01.977 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:01 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: [24/Mar/2024:11:22:01] ENGINE Bus STARTED 2024-03-24T11:22:02.269 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:01 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:22:01.914+0000 7fcccb69c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-24T11:22:02.270 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:01 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:22:01.997+0000 7fcccb69c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-24T11:22:02.270 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:02 smithi139 ceph-mon[25719]: Active manager daemon y restarted 2024-03-24T11:22:02.270 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:02 smithi139 ceph-mon[25719]: Activating manager daemon y 2024-03-24T11:22:02.270 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:02 smithi139 ceph-mon[25719]: osdmap e107: 8 total, 8 up, 8 in 2024-03-24T11:22:02.270 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:02 smithi139 ceph-mon[25719]: from='mgr.? 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-24T11:22:02.270 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:02 smithi139 ceph-mon[25719]: from='mgr.? 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T11:22:02.270 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:02 smithi139 ceph-mon[25719]: from='mgr.? 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-24T11:22:02.270 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:02 smithi139 ceph-mon[25719]: from='mgr.? 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T11:22:02.285 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:02 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: [24/Mar/2024:11:22:02] ENGINE Bus STOPPING 2024-03-24T11:22:02.285 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:02 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: [24/Mar/2024:11:22:02] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-03-24T11:22:02.285 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:02 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: [24/Mar/2024:11:22:02] ENGINE Bus STOPPED 2024-03-24T11:22:02.285 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:02 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: [24/Mar/2024:11:22:02] ENGINE Bus STARTING 2024-03-24T11:22:02.286 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:02 smithi062 ceph-mon[24799]: Active manager daemon y restarted 2024-03-24T11:22:02.286 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:02 smithi062 ceph-mon[24799]: Activating manager daemon y 2024-03-24T11:22:02.286 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:02 smithi062 ceph-mon[24799]: osdmap e107: 8 total, 8 up, 8 in 2024-03-24T11:22:02.286 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:02 smithi062 ceph-mon[24799]: from='mgr.? 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-24T11:22:02.286 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:02 smithi062 ceph-mon[24799]: from='mgr.? 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T11:22:02.286 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:02 smithi062 ceph-mon[24799]: from='mgr.? 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-24T11:22:02.286 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:02 smithi062 ceph-mon[24799]: from='mgr.? 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T11:22:02.286 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:02 smithi062 ceph-mon[29666]: Active manager daemon y restarted 2024-03-24T11:22:02.286 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:02 smithi062 ceph-mon[29666]: Activating manager daemon y 2024-03-24T11:22:02.287 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:02 smithi062 ceph-mon[29666]: osdmap e107: 8 total, 8 up, 8 in 2024-03-24T11:22:02.287 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:02 smithi062 ceph-mon[29666]: from='mgr.? 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-24T11:22:02.287 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:02 smithi062 ceph-mon[29666]: from='mgr.? 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T11:22:02.287 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:02 smithi062 ceph-mon[29666]: from='mgr.? 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-24T11:22:02.287 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:02 smithi062 ceph-mon[29666]: from='mgr.? 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T11:22:02.560 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:02 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:22:02.268+0000 7fcccb69c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-24T11:22:02.561 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:02 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:22:02.378+0000 7fcccb69c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-24T11:22:02.561 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:02 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:22:02.465+0000 7fcccb69c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-24T11:22:02.561 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:02 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:22:02.559+0000 7fcccb69c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-24T11:22:02.608 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:02 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: [24/Mar/2024:11:22:02] ENGINE Serving on http://:::9283 2024-03-24T11:22:02.609 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:02 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: [24/Mar/2024:11:22:02] ENGINE Bus STARTED 2024-03-24T11:22:02.857 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:02 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:22:02.739+0000 7fcccb69c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-24T11:22:03.121 INFO:teuthology.orchestra.run.smithi062.stdout:true 2024-03-24T11:22:03.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[29666]: mgrmap e29: y(active, starting, since 0.447928s), standbys: x 2024-03-24T11:22:03.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T11:22:03.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T11:22:03.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:22:03.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-24T11:22:03.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-24T11:22:03.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T11:22:03.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T11:22:03.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T11:22:03.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T11:22:03.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T11:22:03.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T11:22:03.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T11:22:03.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T11:22:03.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-24T11:22:03.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-24T11:22:03.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-24T11:22:03.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[29666]: Manager daemon y is now available 2024-03-24T11:22:03.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:03.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:03.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi062", "caps": []}]: dispatch 2024-03-24T11:22:03.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi139", "caps": []}]: dispatch 2024-03-24T11:22:03.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:03.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:22:03.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-24T11:22:03.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-24T11:22:03.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi062", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:22:03.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:03.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[24799]: mgrmap e29: y(active, starting, since 0.447928s), standbys: x 2024-03-24T11:22:03.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T11:22:03.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T11:22:03.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:22:03.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-24T11:22:03.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-24T11:22:03.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T11:22:03.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T11:22:03.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T11:22:03.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T11:22:03.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T11:22:03.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T11:22:03.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T11:22:03.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T11:22:03.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-24T11:22:03.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-24T11:22:03.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-24T11:22:03.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[24799]: Manager daemon y is now available 2024-03-24T11:22:03.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:03.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:03.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi062", "caps": []}]: dispatch 2024-03-24T11:22:03.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi139", "caps": []}]: dispatch 2024-03-24T11:22:03.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:03.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:22:03.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-24T11:22:03.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-24T11:22:03.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi062", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:22:03.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:03.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:03 smithi139 ceph-mon[25719]: mgrmap e29: y(active, starting, since 0.447928s), standbys: x 2024-03-24T11:22:03.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T11:22:03.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T11:22:03.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:22:03.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-24T11:22:03.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-24T11:22:03.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T11:22:03.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T11:22:03.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T11:22:03.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T11:22:03.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T11:22:03.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T11:22:03.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T11:22:03.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T11:22:03.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-24T11:22:03.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-24T11:22:03.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-24T11:22:03.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:03 smithi139 ceph-mon[25719]: Manager daemon y is now available 2024-03-24T11:22:03.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:03.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:03.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi062", "caps": []}]: dispatch 2024-03-24T11:22:03.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi139", "caps": []}]: dispatch 2024-03-24T11:22:03.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:03.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:22:03.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-24T11:22:03.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-24T11:22:03.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi062", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:22:03.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:03.881 INFO:teuthology.orchestra.run.smithi062.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T11:22:03.881 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi062 smithi062 error 18s ago 11m - - 2024-03-24T11:22:03.881 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi139 smithi139 running 5s ago 11m - - 2024-03-24T11:22:03.881 INFO:teuthology.orchestra.run.smithi062.stdout:alertmanager.a smithi062 *:9093,9094 running (79s) 18s ago 6m 17.2M - 0.23.0 ba2b418f427c 818298d58140 2024-03-24T11:22:03.881 INFO:teuthology.orchestra.run.smithi062.stdout:grafana.a smithi139 *:3000 running (38s) 5s ago 5m 45.5M - 8.3.5 dad864ee21e9 b48167e7171e 2024-03-24T11:22:03.881 INFO:teuthology.orchestra.run.smithi062.stdout:iscsi.foo.smithi062.dggsvk smithi062 starting - - - - 2024-03-24T11:22:03.881 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.x smithi139 *:8443,9283 running (108s) 5s ago 10m 508M - 19.0.0-1667-gdb0330b1 4f325f8440e9 ed2c6bf552c0 2024-03-24T11:22:03.881 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.y smithi062 *:8443,9283,8765 starting - - - - 2024-03-24T11:22:03.881 INFO:teuthology.orchestra.run.smithi062.stdout:mon.a smithi062 running (12m) 18s ago 12m 54.8M 2048M 17.2.0 e1d6a67b021e 712f01aa9180 2024-03-24T11:22:03.882 INFO:teuthology.orchestra.run.smithi062.stdout:mon.b smithi139 running (10m) 5s ago 10m 51.9M 2048M 17.2.0 e1d6a67b021e 8945cdb4e6aa 2024-03-24T11:22:03.882 INFO:teuthology.orchestra.run.smithi062.stdout:mon.c smithi062 running (11m) 18s ago 11m 44.0M 2048M 17.2.0 e1d6a67b021e 39110a16ab62 2024-03-24T11:22:03.882 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.a smithi062 *:9100 running (90s) 18s ago 6m 9.84M - 1.3.1 1dbe0e931976 79d5a02a999f 2024-03-24T11:22:03.882 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.b smithi139 *:9100 running (53s) 5s ago 6m 8795k - 1.3.1 1dbe0e931976 8f9f4608e465 2024-03-24T11:22:03.882 INFO:teuthology.orchestra.run.smithi062.stdout:osd.0 smithi062 running (10m) 18s ago 10m 52.1M 4096M 17.2.0 e1d6a67b021e 8040b30d7170 2024-03-24T11:22:03.882 INFO:teuthology.orchestra.run.smithi062.stdout:osd.1 smithi062 running (10m) 18s ago 10m 53.8M 4096M 17.2.0 e1d6a67b021e e62d22e5dc29 2024-03-24T11:22:03.882 INFO:teuthology.orchestra.run.smithi062.stdout:osd.2 smithi062 running (9m) 18s ago 9m 47.4M 4096M 17.2.0 e1d6a67b021e 714f550103a7 2024-03-24T11:22:03.882 INFO:teuthology.orchestra.run.smithi062.stdout:osd.3 smithi062 running (9m) 18s ago 9m 51.7M 4096M 17.2.0 e1d6a67b021e 1d2868bd5195 2024-03-24T11:22:03.882 INFO:teuthology.orchestra.run.smithi062.stdout:osd.4 smithi139 running (8m) 5s ago 8m 56.3M 4096M 17.2.0 e1d6a67b021e 7568bfa9e74d 2024-03-24T11:22:03.882 INFO:teuthology.orchestra.run.smithi062.stdout:osd.5 smithi139 running (8m) 5s ago 8m 49.8M 4096M 17.2.0 e1d6a67b021e dd0abd599a14 2024-03-24T11:22:03.882 INFO:teuthology.orchestra.run.smithi062.stdout:osd.6 smithi139 running (8m) 5s ago 8m 49.9M 4096M 17.2.0 e1d6a67b021e a4b1289aca69 2024-03-24T11:22:03.882 INFO:teuthology.orchestra.run.smithi062.stdout:osd.7 smithi139 running (7m) 5s ago 7m 51.7M 4096M 17.2.0 e1d6a67b021e bc7fadbc1384 2024-03-24T11:22:03.882 INFO:teuthology.orchestra.run.smithi062.stdout:prometheus.a smithi139 *:9095 running (58s) 5s ago 7m 55.7M - 2.33.4 514e6a882f6e 5c0c71f4d276 2024-03-24T11:22:03.883 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi062.sacpds smithi062 *:8000 running (5m) 18s ago 5m 71.2M - 17.2.0 e1d6a67b021e 89f4ac308e98 2024-03-24T11:22:03.883 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi139.yljesh smithi139 *:8000 running (5m) 5s ago 5m 70.4M - 17.2.0 e1d6a67b021e e189bdc12cae 2024-03-24T11:22:03.883 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi062.xvpxnk smithi062 *:80 running (5m) 18s ago 5m 71.1M - 17.2.0 e1d6a67b021e d1aa74adab44 2024-03-24T11:22:03.883 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi139.pbtrmp smithi139 *:80 running (5m) 5s ago 4m 74.0M - 17.2.0 e1d6a67b021e 7f8e336755d7 2024-03-24T11:22:03.965 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:03 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:22:03.764+0000 7fcccb69c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-24T11:22:03.965 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:03 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:22:03.911+0000 7fcccb69c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-24T11:22:04.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:04 smithi062 ceph-mon[24799]: [24/Mar/2024:11:22:03] ENGINE Bus STARTING 2024-03-24T11:22:04.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:04 smithi062 ceph-mon[24799]: mgrmap e30: y(active, since 1.4588s), standbys: x 2024-03-24T11:22:04.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:04 smithi062 ceph-mon[24799]: from='client.25291 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:22:04.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:04 smithi062 ceph-mon[24799]: pgmap v3: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:22:04.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:04 smithi062 ceph-mon[24799]: [24/Mar/2024:11:22:03] ENGINE Serving on https://172.21.15.62:7150 2024-03-24T11:22:04.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:04 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi139", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:22:04.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:04 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:04.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:04 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:04.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:04 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:04.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:04 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:04.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:04 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:04.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:04 smithi062 ceph-mon[29666]: [24/Mar/2024:11:22:03] ENGINE Bus STARTING 2024-03-24T11:22:04.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:04 smithi062 ceph-mon[29666]: mgrmap e30: y(active, since 1.4588s), standbys: x 2024-03-24T11:22:04.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:04 smithi062 ceph-mon[29666]: from='client.25291 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:22:04.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:04 smithi062 ceph-mon[29666]: pgmap v3: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:22:04.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:04 smithi062 ceph-mon[29666]: [24/Mar/2024:11:22:03] ENGINE Serving on https://172.21.15.62:7150 2024-03-24T11:22:04.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:04 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi139", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:22:04.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:04 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:04.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:04 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:04.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:04 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:04.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:04 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:04.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:04 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:04.228 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:04 smithi139 ceph-mon[25719]: [24/Mar/2024:11:22:03] ENGINE Bus STARTING 2024-03-24T11:22:04.229 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:04 smithi139 ceph-mon[25719]: mgrmap e30: y(active, since 1.4588s), standbys: x 2024-03-24T11:22:04.229 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:04 smithi139 ceph-mon[25719]: from='client.25291 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:22:04.229 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:04 smithi139 ceph-mon[25719]: pgmap v3: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:22:04.229 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:04 smithi139 ceph-mon[25719]: [24/Mar/2024:11:22:03] ENGINE Serving on https://172.21.15.62:7150 2024-03-24T11:22:04.229 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:04 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi139", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:22:04.229 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:04 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:04.229 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:04 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:04.229 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:04 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:04.229 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:04 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:04.229 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:04 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:04.338 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:22:04.339 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": { 2024-03-24T11:22:04.339 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 3 2024-03-24T11:22:04.339 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:22:04.339 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": { 2024-03-24T11:22:04.339 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-24T11:22:04.339 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:22:04.339 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": { 2024-03-24T11:22:04.339 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 8 2024-03-24T11:22:04.339 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:22:04.339 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": {}, 2024-03-24T11:22:04.339 INFO:teuthology.orchestra.run.smithi062.stdout: "rgw": { 2024-03-24T11:22:04.339 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4 2024-03-24T11:22:04.339 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:22:04.339 INFO:teuthology.orchestra.run.smithi062.stdout: "overall": { 2024-03-24T11:22:04.339 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 15, 2024-03-24T11:22:04.340 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-24T11:22:04.340 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-24T11:22:04.340 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:22:04.590 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:04 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:22:04.281+0000 7fcccb69c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-24T11:22:04.590 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:04 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:22:04.379+0000 7fcccb69c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-24T11:22:04.590 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:04 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:22:04.589+0000 7fcccb69c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-24T11:22:04.742 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:22:04.743 INFO:teuthology.orchestra.run.smithi062.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69", 2024-03-24T11:22:04.743 INFO:teuthology.orchestra.run.smithi062.stdout: "in_progress": true, 2024-03-24T11:22:04.743 INFO:teuthology.orchestra.run.smithi062.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T11:22:04.743 INFO:teuthology.orchestra.run.smithi062.stdout: "services_complete": [], 2024-03-24T11:22:04.744 INFO:teuthology.orchestra.run.smithi062.stdout: "progress": "", 2024-03-24T11:22:04.744 INFO:teuthology.orchestra.run.smithi062.stdout: "message": "", 2024-03-24T11:22:04.744 INFO:teuthology.orchestra.run.smithi062.stdout: "is_paused": false 2024-03-24T11:22:04.744 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:22:04.857 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:04 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:22:04.784+0000 7fcccb69c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-24T11:22:05.142 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:05 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:22:05.077+0000 7fcccb69c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-24T11:22:05.143 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:05 smithi139 ceph-mon[25719]: Updating smithi062:/etc/ceph/ceph.conf 2024-03-24T11:22:05.143 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:05 smithi139 ceph-mon[25719]: Updating smithi139:/etc/ceph/ceph.conf 2024-03-24T11:22:05.143 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:05 smithi139 ceph-mon[25719]: [24/Mar/2024:11:22:03] ENGINE Serving on http://172.21.15.62:8765 2024-03-24T11:22:05.143 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:05 smithi139 ceph-mon[25719]: [24/Mar/2024:11:22:03] ENGINE Bus STARTED 2024-03-24T11:22:05.143 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:05 smithi139 ceph-mon[25719]: from='client.25312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:22:05.143 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:05 smithi139 ceph-mon[25719]: Updating smithi139:/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T11:22:05.143 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:05 smithi139 ceph-mon[25719]: Updating smithi062:/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T11:22:05.143 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:05 smithi139 ceph-mon[25719]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:22:05.143 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:05 smithi139 ceph-mon[25719]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2024-03-24T11:22:05.143 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:05 smithi139 ceph-mon[25719]: Updating smithi139:/etc/ceph/ceph.client.admin.keyring 2024-03-24T11:22:05.144 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:05 smithi139 ceph-mon[25719]: pgmap v4: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:22:05.144 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:05 smithi139 ceph-mon[25719]: mgrmap e31: y(active, since 2s), standbys: x 2024-03-24T11:22:05.144 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:05 smithi139 ceph-mon[25719]: Updating smithi062:/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T11:22:05.144 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:05 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/1613656278' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:05.144 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:05.144 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:05.144 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:05.144 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:05.144 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:05.144 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:05.144 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi062.dggsvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-03-24T11:22:05.144 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:05.192 INFO:teuthology.orchestra.run.smithi062.stdout:HEALTH_OK 2024-03-24T11:22:05.470 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:05 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:22:05.378+0000 7fcccb69c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-24T11:22:05.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:05 smithi062 ceph-mon[29666]: Updating smithi062:/etc/ceph/ceph.conf 2024-03-24T11:22:05.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:05 smithi062 ceph-mon[29666]: Updating smithi139:/etc/ceph/ceph.conf 2024-03-24T11:22:05.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:05 smithi062 ceph-mon[29666]: [24/Mar/2024:11:22:03] ENGINE Serving on http://172.21.15.62:8765 2024-03-24T11:22:05.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:05 smithi062 ceph-mon[29666]: [24/Mar/2024:11:22:03] ENGINE Bus STARTED 2024-03-24T11:22:05.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:05 smithi062 ceph-mon[29666]: from='client.25312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:22:05.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:05 smithi062 ceph-mon[29666]: Updating smithi139:/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T11:22:05.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:05 smithi062 ceph-mon[29666]: Updating smithi062:/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T11:22:05.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:05 smithi062 ceph-mon[29666]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:22:05.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:05 smithi062 ceph-mon[29666]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2024-03-24T11:22:05.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:05 smithi062 ceph-mon[29666]: Updating smithi139:/etc/ceph/ceph.client.admin.keyring 2024-03-24T11:22:05.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:05 smithi062 ceph-mon[29666]: pgmap v4: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:22:05.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:05 smithi062 ceph-mon[29666]: mgrmap e31: y(active, since 2s), standbys: x 2024-03-24T11:22:05.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:05 smithi062 ceph-mon[29666]: Updating smithi062:/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T11:22:05.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:05 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/1613656278' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:05.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:05.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:05.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:05.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:05.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:05.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:05.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi062.dggsvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-03-24T11:22:05.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:05.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:05 smithi062 ceph-mon[24799]: Updating smithi062:/etc/ceph/ceph.conf 2024-03-24T11:22:05.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:05 smithi062 ceph-mon[24799]: Updating smithi139:/etc/ceph/ceph.conf 2024-03-24T11:22:05.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:05 smithi062 ceph-mon[24799]: [24/Mar/2024:11:22:03] ENGINE Serving on http://172.21.15.62:8765 2024-03-24T11:22:05.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:05 smithi062 ceph-mon[24799]: [24/Mar/2024:11:22:03] ENGINE Bus STARTED 2024-03-24T11:22:05.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:05 smithi062 ceph-mon[24799]: from='client.25312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:22:05.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:05 smithi062 ceph-mon[24799]: Updating smithi139:/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T11:22:05.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:05 smithi062 ceph-mon[24799]: Updating smithi062:/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T11:22:05.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:05 smithi062 ceph-mon[24799]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:22:05.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:05 smithi062 ceph-mon[24799]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2024-03-24T11:22:05.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:05 smithi062 ceph-mon[24799]: Updating smithi139:/etc/ceph/ceph.client.admin.keyring 2024-03-24T11:22:05.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:05 smithi062 ceph-mon[24799]: pgmap v4: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:22:05.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:05 smithi062 ceph-mon[24799]: mgrmap e31: y(active, since 2s), standbys: x 2024-03-24T11:22:05.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:05 smithi062 ceph-mon[24799]: Updating smithi062:/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T11:22:05.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:05 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/1613656278' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:05.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:05.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:05.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:05.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:05.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:05.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:05.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi062.dggsvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-03-24T11:22:05.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:05.857 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:05 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:22:05.639+0000 7fcccb69c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-24T11:22:06.312 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:06 smithi139 ceph-mon[25719]: Updating smithi139:/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T11:22:06.312 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:06 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:06.312 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:06 smithi139 ceph-mon[25719]: Reconfiguring iscsi.foo.smithi062.dggsvk (dependencies changed)... 2024-03-24T11:22:06.313 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:06 smithi139 ceph-mon[25719]: Reconfiguring daemon iscsi.foo.smithi062.dggsvk on smithi062 2024-03-24T11:22:06.313 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:06 smithi139 ceph-mon[25719]: from='client.15435 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:22:06.313 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:06 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/2763603388' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:22:06.313 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:06.313 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:06.313 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:06.313 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:06.313 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:06.313 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:06.313 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:06.313 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:06.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:06 smithi062 ceph-mon[24799]: Updating smithi139:/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T11:22:06.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:06 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:06.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:06 smithi062 ceph-mon[24799]: Reconfiguring iscsi.foo.smithi062.dggsvk (dependencies changed)... 2024-03-24T11:22:06.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:06 smithi062 ceph-mon[24799]: Reconfiguring daemon iscsi.foo.smithi062.dggsvk on smithi062 2024-03-24T11:22:06.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:06 smithi062 ceph-mon[24799]: from='client.15435 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:22:06.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:06 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/2763603388' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:22:06.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:06.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:06.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:06.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:06.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:06.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:06.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:06.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:06.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:06 smithi062 ceph-mon[29666]: Updating smithi139:/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T11:22:06.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:06 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:06.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:06 smithi062 ceph-mon[29666]: Reconfiguring iscsi.foo.smithi062.dggsvk (dependencies changed)... 2024-03-24T11:22:06.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:06 smithi062 ceph-mon[29666]: Reconfiguring daemon iscsi.foo.smithi062.dggsvk on smithi062 2024-03-24T11:22:06.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:06 smithi062 ceph-mon[29666]: from='client.15435 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:22:06.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:06 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/2763603388' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:22:06.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:06.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:06.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:06.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:06.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:06.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:06.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:06.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:06.591 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:06 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:22:06.523+0000 7fcccb69c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-24T11:22:06.591 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:06 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[55656]: ts=2024-03-24T11:22:06.324Z caller=refresh.go:98 level=error component="discovery manager scrape" discovery=http msg="Unable to refresh target groups" err="Get \"http://172.21.15.139:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.139:8765: connect: connection refused" 2024-03-24T11:22:06.592 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:06 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[55656]: ts=2024-03-24T11:22:06.325Z caller=refresh.go:98 level=error component="discovery manager scrape" discovery=http msg="Unable to refresh target groups" err="Get \"http://172.21.15.139:8765/sd/prometheus/sd-config?service=node-exporter\": dial tcp 172.21.15.139:8765: connect: connection refused" 2024-03-24T11:22:06.592 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:06 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[55656]: ts=2024-03-24T11:22:06.326Z caller=refresh.go:98 level=error component="discovery manager scrape" discovery=http msg="Unable to refresh target groups" err="Get \"http://172.21.15.139:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.139:8765: connect: connection refused" 2024-03-24T11:22:06.592 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:06 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[55656]: ts=2024-03-24T11:22:06.326Z caller=refresh.go:98 level=error component="discovery manager notify" discovery=http msg="Unable to refresh target groups" err="Get \"http://172.21.15.139:8765/sd/prometheus/sd-config?service=alertmanager\": dial tcp 172.21.15.139:8765: connect: connection refused" 2024-03-24T11:22:07.305 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:07 smithi139 ceph-mon[25719]: pgmap v5: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:22:07.305 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:07 smithi139 ceph-mon[25719]: mgrmap e32: y(active, since 4s), standbys: x 2024-03-24T11:22:07.305 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:07 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:07.305 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:07 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:07.305 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:07 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:07.306 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:07 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:22:07.158+0000 7fcccb69c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-24T11:22:07.306 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:07 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:22:07.266+0000 7fcccb69c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-24T11:22:07.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:07 smithi062 ceph-mon[24799]: pgmap v5: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:22:07.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:07 smithi062 ceph-mon[24799]: mgrmap e32: y(active, since 4s), standbys: x 2024-03-24T11:22:07.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:07 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:07.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:07 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:07.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:07 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:07.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:07 smithi062 ceph-mon[29666]: pgmap v5: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:22:07.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:07 smithi062 ceph-mon[29666]: mgrmap e32: y(active, since 4s), standbys: x 2024-03-24T11:22:07.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:07 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:07.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:07 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:07.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:07 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:07.608 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:07 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:22:07.501+0000 7fcccb69c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-24T11:22:07.608 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:07 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: [24/Mar/2024:11:22:07] ENGINE Bus STARTING 2024-03-24T11:22:07.900 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:07 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: CherryPy Checker: 2024-03-24T11:22:07.900 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:07 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: The Application mounted at '' has an empty config. 2024-03-24T11:22:07.900 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:07 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: 2024-03-24T11:22:07.900 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:07 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: [24/Mar/2024:11:22:07] ENGINE Serving on http://:::9283 2024-03-24T11:22:07.900 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:07 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: [24/Mar/2024:11:22:07] ENGINE Bus STARTED 2024-03-24T11:22:08.276 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:08 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:08.276 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:08 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:08.276 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:08 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:08.276 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:08 smithi062 ceph-mon[29666]: Standby manager daemon x restarted 2024-03-24T11:22:08.276 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:08 smithi062 ceph-mon[29666]: Standby manager daemon x started 2024-03-24T11:22:08.276 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:08 smithi062 ceph-mon[29666]: from='mgr.? 172.21.15.139:0/64195946' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-24T11:22:08.276 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:08 smithi062 ceph-mon[29666]: from='mgr.? 172.21.15.139:0/64195946' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T11:22:08.276 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:08 smithi062 ceph-mon[29666]: from='mgr.? 172.21.15.139:0/64195946' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-24T11:22:08.277 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:08 smithi062 ceph-mon[29666]: from='mgr.? 172.21.15.139:0/64195946' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T11:22:08.277 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:08 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:08.277 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:08 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:08.277 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:08 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:08.277 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:08 smithi062 ceph-mon[24799]: Standby manager daemon x restarted 2024-03-24T11:22:08.277 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:08 smithi062 ceph-mon[24799]: Standby manager daemon x started 2024-03-24T11:22:08.277 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:08 smithi062 ceph-mon[24799]: from='mgr.? 172.21.15.139:0/64195946' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-24T11:22:08.277 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:08 smithi062 ceph-mon[24799]: from='mgr.? 172.21.15.139:0/64195946' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T11:22:08.277 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:08 smithi062 ceph-mon[24799]: from='mgr.? 172.21.15.139:0/64195946' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-24T11:22:08.277 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:08 smithi062 ceph-mon[24799]: from='mgr.? 172.21.15.139:0/64195946' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T11:22:08.470 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:08 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:08.471 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:08 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:08.471 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:08 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:08.471 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:08 smithi139 ceph-mon[25719]: Standby manager daemon x restarted 2024-03-24T11:22:08.471 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:08 smithi139 ceph-mon[25719]: Standby manager daemon x started 2024-03-24T11:22:08.471 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:08 smithi139 ceph-mon[25719]: from='mgr.? 172.21.15.139:0/64195946' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-24T11:22:08.471 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:08 smithi139 ceph-mon[25719]: from='mgr.? 172.21.15.139:0/64195946' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T11:22:08.471 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:08 smithi139 ceph-mon[25719]: from='mgr.? 172.21.15.139:0/64195946' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-24T11:22:08.471 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:08 smithi139 ceph-mon[25719]: from='mgr.? 172.21.15.139:0/64195946' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T11:22:08.762 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:08 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[55656]: ts=2024-03-24T11:22:08.659Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"smithi139:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"172.21.15.139:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-24T11:22:09.291 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:09 smithi062 ceph-mon[24799]: pgmap v6: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:22:09.291 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:09 smithi062 ceph-mon[24799]: mgrmap e33: y(active, since 6s), standbys: x 2024-03-24T11:22:09.291 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:09 smithi062 ceph-mon[29666]: pgmap v6: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:22:09.291 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:09 smithi062 ceph-mon[29666]: mgrmap e33: y(active, since 6s), standbys: x 2024-03-24T11:22:09.607 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:09 smithi139 ceph-mon[25719]: pgmap v6: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:22:09.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:09 smithi139 ceph-mon[25719]: mgrmap e33: y(active, since 6s), standbys: x 2024-03-24T11:22:09.608 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:09 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[52317]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:22:09] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-03-24T11:22:10.477 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:22:10.256+0000 7f95852cc700 -1 log_channel(cephadm) log [ERR] : cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi062.dggsvk ... 2024-03-24T11:22:10.477 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: Creating ceph-iscsi config... 2024-03-24T11:22:10.477 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/iscsi-gateway.cfg 2024-03-24T11:22:10.477 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/tcmu-runner-entrypoint.sh 2024-03-24T11:22:10.477 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: Non-zero exit code 1 from systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk 2024-03-24T11:22:10.477 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: systemctl: stderr Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:22:10.477 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: systemctl: stderr See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:22:10.477 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: Traceback (most recent call last): 2024-03-24T11:22:10.478 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-24T11:22:10.478 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: return _run_code(code, main_globals, None, 2024-03-24T11:22:10.478 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-24T11:22:10.478 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: exec(code, run_globals) 2024-03-24T11:22:10.478 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5520, in 2024-03-24T11:22:10.478 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5508, in main 2024-03-24T11:22:10.478 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3038, in command_deploy_from 2024-03-24T11:22:10.478 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3073, in _common_deploy 2024-03-24T11:22:10.478 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3093, in _deploy_daemon_container 2024-03-24T11:22:10.478 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 1065, in deploy_daemon 2024-03-24T11:22:10.478 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-03-24T11:22:10.479 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: RuntimeError: Failed command: systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk: Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:22:10.479 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:22:10.479 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: Traceback (most recent call last): 2024-03-24T11:22:10.479 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1109, in _check_daemons 2024-03-24T11:22:10.479 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: self.mgr._daemon_action(daemon_spec, action=action) 2024-03-24T11:22:10.479 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/usr/share/ceph/mgr/cephadm/module.py", line 2372, in _daemon_action 2024-03-24T11:22:10.479 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: CephadmServe(self)._create_daemon(daemon_spec, reconfig=(action == 'reconfig'))) 2024-03-24T11:22:10.479 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/usr/share/ceph/mgr/cephadm/module.py", line 739, in wait_async 2024-03-24T11:22:10.480 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: return self.event_loop.get_result(coro, timeout) 2024-03-24T11:22:10.480 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/usr/share/ceph/mgr/cephadm/ssh.py", line 64, in get_result 2024-03-24T11:22:10.480 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: return future.result(timeout) 2024-03-24T11:22:10.480 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/lib64/python3.6/concurrent/futures/_base.py", line 432, in result 2024-03-24T11:22:10.480 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: return self.__get_result() 2024-03-24T11:22:10.480 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/lib64/python3.6/concurrent/futures/_base.py", line 384, in __get_result 2024-03-24T11:22:10.480 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: raise self._exception 2024-03-24T11:22:10.480 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1369, in _create_daemon 2024-03-24T11:22:10.480 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: config_blobs=daemon_spec.final_config, 2024-03-24T11:22:10.480 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1671, in _run_cephadm 2024-03-24T11:22:10.480 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: f'cephadm exited with an error code: {code}, stderr: {err}') 2024-03-24T11:22:10.480 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: orchestrator._interface.OrchestratorError: cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi062.dggsvk ... 2024-03-24T11:22:10.480 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: Creating ceph-iscsi config... 2024-03-24T11:22:10.480 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/iscsi-gateway.cfg 2024-03-24T11:22:10.481 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/tcmu-runner-entrypoint.sh 2024-03-24T11:22:10.481 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: Non-zero exit code 1 from systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk 2024-03-24T11:22:10.481 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: systemctl: stderr Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:22:10.481 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: systemctl: stderr See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:22:10.481 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: Traceback (most recent call last): 2024-03-24T11:22:10.481 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-24T11:22:10.481 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: return _run_code(code, main_globals, None, 2024-03-24T11:22:10.481 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-24T11:22:10.481 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: exec(code, run_globals) 2024-03-24T11:22:10.481 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5520, in 2024-03-24T11:22:10.481 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5508, in main 2024-03-24T11:22:10.481 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3038, in command_deploy_from 2024-03-24T11:22:10.481 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3073, in _common_deploy 2024-03-24T11:22:10.481 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3093, in _deploy_daemon_container 2024-03-24T11:22:10.482 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 1065, in deploy_daemon 2024-03-24T11:22:10.482 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-03-24T11:22:10.482 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: RuntimeError: Failed command: systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk: Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:22:10.482 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:22:11.169 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:11 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:11.169 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:11 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:11.169 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:11 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:11.169 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:11 smithi062 ceph-mon[24799]: pgmap v7: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-03-24T11:22:11.170 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:11 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:11.170 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:11 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:11.170 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:11 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:11.170 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:11 smithi062 ceph-mon[29666]: pgmap v7: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-03-24T11:22:11.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:11 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:11.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:11 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:11.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:11 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:11.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:11 smithi139 ceph-mon[25719]: pgmap v7: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-03-24T11:22:12.185 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:12 smithi062 ceph-mon[24799]: cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi062.dggsvk ... 2024-03-24T11:22:12.185 INFO:journalctl@ceph.mon.a.smithi062.stdout: Creating ceph-iscsi config... 2024-03-24T11:22:12.185 INFO:journalctl@ceph.mon.a.smithi062.stdout: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/iscsi-gateway.cfg 2024-03-24T11:22:12.185 INFO:journalctl@ceph.mon.a.smithi062.stdout: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/tcmu-runner-entrypoint.sh 2024-03-24T11:22:12.185 INFO:journalctl@ceph.mon.a.smithi062.stdout: Non-zero exit code 1 from systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk 2024-03-24T11:22:12.185 INFO:journalctl@ceph.mon.a.smithi062.stdout: systemctl: stderr Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:22:12.185 INFO:journalctl@ceph.mon.a.smithi062.stdout: systemctl: stderr See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:22:12.185 INFO:journalctl@ceph.mon.a.smithi062.stdout: Traceback (most recent call last): 2024-03-24T11:22:12.185 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-24T11:22:12.185 INFO:journalctl@ceph.mon.a.smithi062.stdout: return _run_code(code, main_globals, None, 2024-03-24T11:22:12.185 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-24T11:22:12.185 INFO:journalctl@ceph.mon.a.smithi062.stdout: exec(code, run_globals) 2024-03-24T11:22:12.186 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5520, in 2024-03-24T11:22:12.186 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5508, in main 2024-03-24T11:22:12.186 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3038, in command_deploy_from 2024-03-24T11:22:12.186 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3073, in _common_deploy 2024-03-24T11:22:12.186 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3093, in _deploy_daemon_container 2024-03-24T11:22:12.186 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 1065, in deploy_daemon 2024-03-24T11:22:12.186 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-03-24T11:22:12.186 INFO:journalctl@ceph.mon.a.smithi062.stdout: RuntimeError: Failed command: systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk: Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:22:12.186 INFO:journalctl@ceph.mon.a.smithi062.stdout: See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:22:12.186 INFO:journalctl@ceph.mon.a.smithi062.stdout: Traceback (most recent call last): 2024-03-24T11:22:12.186 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1109, in _check_daemons 2024-03-24T11:22:12.186 INFO:journalctl@ceph.mon.a.smithi062.stdout: self.mgr._daemon_action(daemon_spec, action=action) 2024-03-24T11:22:12.186 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/usr/share/ceph/mgr/cephadm/module.py", line 2372, in _daemon_action 2024-03-24T11:22:12.187 INFO:journalctl@ceph.mon.a.smithi062.stdout: CephadmServe(self)._create_daemon(daemon_spec, reconfig=(action == 'reconfig'))) 2024-03-24T11:22:12.187 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/usr/share/ceph/mgr/cephadm/module.py", line 739, in wait_async 2024-03-24T11:22:12.187 INFO:journalctl@ceph.mon.a.smithi062.stdout: return self.event_loop.get_result(coro, timeout) 2024-03-24T11:22:12.187 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/usr/share/ceph/mgr/cephadm/ssh.py", line 64, in get_result 2024-03-24T11:22:12.187 INFO:journalctl@ceph.mon.a.smithi062.stdout: return future.result(timeout) 2024-03-24T11:22:12.187 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib64/python3.6/concurrent/futures/_base.py", line 432, in result 2024-03-24T11:22:12.187 INFO:journalctl@ceph.mon.a.smithi062.stdout: return self.__get_result() 2024-03-24T11:22:12.187 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib64/python3.6/concurrent/futures/_base.py", line 384, in __get_result 2024-03-24T11:22:12.187 INFO:journalctl@ceph.mon.a.smithi062.stdout: raise self._exception 2024-03-24T11:22:12.187 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1369, in _create_daemon 2024-03-24T11:22:12.187 INFO:journalctl@ceph.mon.a.smithi062.stdout: config_blobs=daemon_spec.final_config, 2024-03-24T11:22:12.187 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1671, in _run_cephadm 2024-03-24T11:22:12.187 INFO:journalctl@ceph.mon.a.smithi062.stdout: f'cephadm exited with an error code: {code}, stderr: {err}') 2024-03-24T11:22:12.187 INFO:journalctl@ceph.mon.a.smithi062.stdout: orchestrator._interface.OrchestratorError: cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi062.dggsvk ... 2024-03-24T11:22:12.187 INFO:journalctl@ceph.mon.a.smithi062.stdout: Creating ceph-iscsi config... 2024-03-24T11:22:12.188 INFO:journalctl@ceph.mon.a.smithi062.stdout: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/iscsi-gateway.cfg 2024-03-24T11:22:12.188 INFO:journalctl@ceph.mon.a.smithi062.stdout: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/tcmu-runner-entrypoint.sh 2024-03-24T11:22:12.188 INFO:journalctl@ceph.mon.a.smithi062.stdout: Non-zero exit code 1 from systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk 2024-03-24T11:22:12.188 INFO:journalctl@ceph.mon.a.smithi062.stdout: systemctl: stderr Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:22:12.188 INFO:journalctl@ceph.mon.a.smithi062.stdout: systemctl: stderr See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:22:12.188 INFO:journalctl@ceph.mon.a.smithi062.stdout: Traceback (most recent call last): 2024-03-24T11:22:12.188 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-24T11:22:12.188 INFO:journalctl@ceph.mon.a.smithi062.stdout: return _run_code(code, main_globals, None, 2024-03-24T11:22:12.188 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-24T11:22:12.188 INFO:journalctl@ceph.mon.a.smithi062.stdout: exec(code, run_globals) 2024-03-24T11:22:12.188 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5520, in 2024-03-24T11:22:12.188 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5508, in main 2024-03-24T11:22:12.188 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3038, in command_deploy_from 2024-03-24T11:22:12.188 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3073, in _common_deploy 2024-03-24T11:22:12.189 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:12 smithi062 ceph-mon[29666]: cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi062.dggsvk ... 2024-03-24T11:22:12.189 INFO:journalctl@ceph.mon.c.smithi062.stdout: Creating ceph-iscsi config... 2024-03-24T11:22:12.189 INFO:journalctl@ceph.mon.c.smithi062.stdout: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/iscsi-gateway.cfg 2024-03-24T11:22:12.189 INFO:journalctl@ceph.mon.c.smithi062.stdout: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/tcmu-runner-entrypoint.sh 2024-03-24T11:22:12.189 INFO:journalctl@ceph.mon.c.smithi062.stdout: Non-zero exit code 1 from systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk 2024-03-24T11:22:12.189 INFO:journalctl@ceph.mon.c.smithi062.stdout: systemctl: stderr Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:22:12.189 INFO:journalctl@ceph.mon.c.smithi062.stdout: systemctl: stderr See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:22:12.189 INFO:journalctl@ceph.mon.c.smithi062.stdout: Traceback (most recent call last): 2024-03-24T11:22:12.189 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-24T11:22:12.189 INFO:journalctl@ceph.mon.c.smithi062.stdout: return _run_code(code, main_globals, None, 2024-03-24T11:22:12.189 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-24T11:22:12.189 INFO:journalctl@ceph.mon.c.smithi062.stdout: exec(code, run_globals) 2024-03-24T11:22:12.189 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5520, in 2024-03-24T11:22:12.190 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5508, in main 2024-03-24T11:22:12.190 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3038, in command_deploy_from 2024-03-24T11:22:12.190 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3073, in _common_deploy 2024-03-24T11:22:12.190 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3093, in _deploy_daemon_container 2024-03-24T11:22:12.190 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 1065, in deploy_daemon 2024-03-24T11:22:12.190 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-03-24T11:22:12.190 INFO:journalctl@ceph.mon.c.smithi062.stdout: RuntimeError: Failed command: systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk: Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:22:12.190 INFO:journalctl@ceph.mon.c.smithi062.stdout: See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:22:12.190 INFO:journalctl@ceph.mon.c.smithi062.stdout: Traceback (most recent call last): 2024-03-24T11:22:12.190 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1109, in _check_daemons 2024-03-24T11:22:12.190 INFO:journalctl@ceph.mon.c.smithi062.stdout: self.mgr._daemon_action(daemon_spec, action=action) 2024-03-24T11:22:12.190 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/usr/share/ceph/mgr/cephadm/module.py", line 2372, in _daemon_action 2024-03-24T11:22:12.190 INFO:journalctl@ceph.mon.c.smithi062.stdout: CephadmServe(self)._create_daemon(daemon_spec, reconfig=(action == 'reconfig'))) 2024-03-24T11:22:12.190 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/usr/share/ceph/mgr/cephadm/module.py", line 739, in wait_async 2024-03-24T11:22:12.191 INFO:journalctl@ceph.mon.c.smithi062.stdout: return self.event_loop.get_result(coro, timeout) 2024-03-24T11:22:12.191 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/usr/share/ceph/mgr/cephadm/ssh.py", line 64, in get_result 2024-03-24T11:22:12.191 INFO:journalctl@ceph.mon.c.smithi062.stdout: return future.result(timeout) 2024-03-24T11:22:12.191 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/lib64/python3.6/concurrent/futures/_base.py", line 432, in result 2024-03-24T11:22:12.191 INFO:journalctl@ceph.mon.c.smithi062.stdout: return self.__get_result() 2024-03-24T11:22:12.191 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/lib64/python3.6/concurrent/futures/_base.py", line 384, in __get_result 2024-03-24T11:22:12.191 INFO:journalctl@ceph.mon.c.smithi062.stdout: raise self._exception 2024-03-24T11:22:12.191 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1369, in _create_daemon 2024-03-24T11:22:12.191 INFO:journalctl@ceph.mon.c.smithi062.stdout: config_blobs=daemon_spec.final_config, 2024-03-24T11:22:12.191 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1671, in _run_cephadm 2024-03-24T11:22:12.191 INFO:journalctl@ceph.mon.c.smithi062.stdout: f'cephadm exited with an error code: {code}, stderr: {err}') 2024-03-24T11:22:12.191 INFO:journalctl@ceph.mon.c.smithi062.stdout: orchestrator._interface.OrchestratorError: cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi062.dggsvk ... 2024-03-24T11:22:12.191 INFO:journalctl@ceph.mon.c.smithi062.stdout: Creating ceph-iscsi config... 2024-03-24T11:22:12.191 INFO:journalctl@ceph.mon.c.smithi062.stdout: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/iscsi-gateway.cfg 2024-03-24T11:22:12.191 INFO:journalctl@ceph.mon.c.smithi062.stdout: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/tcmu-runner-entrypoint.sh 2024-03-24T11:22:12.192 INFO:journalctl@ceph.mon.c.smithi062.stdout: Non-zero exit code 1 from systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk 2024-03-24T11:22:12.192 INFO:journalctl@ceph.mon.c.smithi062.stdout: systemctl: stderr Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:22:12.192 INFO:journalctl@ceph.mon.c.smithi062.stdout: systemctl: stderr See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:22:12.192 INFO:journalctl@ceph.mon.c.smithi062.stdout: Traceback (most recent call last): 2024-03-24T11:22:12.192 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-24T11:22:12.192 INFO:journalctl@ceph.mon.c.smithi062.stdout: return _run_code(code, main_globals, None, 2024-03-24T11:22:12.192 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-24T11:22:12.192 INFO:journalctl@ceph.mon.c.smithi062.stdout: exec(code, run_globals) 2024-03-24T11:22:12.192 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5520, in 2024-03-24T11:22:12.192 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5508, in main 2024-03-24T11:22:12.192 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3038, in command_deploy_from 2024-03-24T11:22:12.192 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3073, in _common_deploy 2024-03-24T11:22:12.193 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3093, in _deploy_daemon_container 2024-03-24T11:22:12.193 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 1065, in deploy_daemon 2024-03-24T11:22:12.193 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-03-24T11:22:12.193 INFO:journalctl@ceph.mon.a.smithi062.stdout: RuntimeError: Failed command: systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk: Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:22:12.193 INFO:journalctl@ceph.mon.a.smithi062.stdout: See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:22:12.193 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:12 smithi062 ceph-mon[24799]: Reconfiguring prometheus.a (dependencies changed)... 2024-03-24T11:22:12.193 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:12 smithi062 ceph-mon[24799]: Reconfiguring daemon prometheus.a on smithi139 2024-03-24T11:22:12.194 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3093, in _deploy_daemon_container 2024-03-24T11:22:12.194 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 1065, in deploy_daemon 2024-03-24T11:22:12.194 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-03-24T11:22:12.194 INFO:journalctl@ceph.mon.c.smithi062.stdout: RuntimeError: Failed command: systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk: Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:22:12.194 INFO:journalctl@ceph.mon.c.smithi062.stdout: See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:22:12.194 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:12 smithi062 ceph-mon[29666]: Reconfiguring prometheus.a (dependencies changed)... 2024-03-24T11:22:12.194 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:12 smithi062 ceph-mon[29666]: Reconfiguring daemon prometheus.a on smithi139 2024-03-24T11:22:12.341 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:12 smithi139 ceph-mon[25719]: cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi062.dggsvk ... 2024-03-24T11:22:12.341 INFO:journalctl@ceph.mon.b.smithi139.stdout: Creating ceph-iscsi config... 2024-03-24T11:22:12.342 INFO:journalctl@ceph.mon.b.smithi139.stdout: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/iscsi-gateway.cfg 2024-03-24T11:22:12.342 INFO:journalctl@ceph.mon.b.smithi139.stdout: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/tcmu-runner-entrypoint.sh 2024-03-24T11:22:12.342 INFO:journalctl@ceph.mon.b.smithi139.stdout: Non-zero exit code 1 from systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk 2024-03-24T11:22:12.342 INFO:journalctl@ceph.mon.b.smithi139.stdout: systemctl: stderr Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:22:12.342 INFO:journalctl@ceph.mon.b.smithi139.stdout: systemctl: stderr See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:22:12.342 INFO:journalctl@ceph.mon.b.smithi139.stdout: Traceback (most recent call last): 2024-03-24T11:22:12.342 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-24T11:22:12.342 INFO:journalctl@ceph.mon.b.smithi139.stdout: return _run_code(code, main_globals, None, 2024-03-24T11:22:12.342 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-24T11:22:12.342 INFO:journalctl@ceph.mon.b.smithi139.stdout: exec(code, run_globals) 2024-03-24T11:22:12.342 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5520, in 2024-03-24T11:22:12.342 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5508, in main 2024-03-24T11:22:12.342 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3038, in command_deploy_from 2024-03-24T11:22:12.343 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3073, in _common_deploy 2024-03-24T11:22:12.343 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3093, in _deploy_daemon_container 2024-03-24T11:22:12.343 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 1065, in deploy_daemon 2024-03-24T11:22:12.343 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-03-24T11:22:12.343 INFO:journalctl@ceph.mon.b.smithi139.stdout: RuntimeError: Failed command: systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk: Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:22:12.343 INFO:journalctl@ceph.mon.b.smithi139.stdout: See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:22:12.343 INFO:journalctl@ceph.mon.b.smithi139.stdout: Traceback (most recent call last): 2024-03-24T11:22:12.343 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1109, in _check_daemons 2024-03-24T11:22:12.343 INFO:journalctl@ceph.mon.b.smithi139.stdout: self.mgr._daemon_action(daemon_spec, action=action) 2024-03-24T11:22:12.343 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/usr/share/ceph/mgr/cephadm/module.py", line 2372, in _daemon_action 2024-03-24T11:22:12.343 INFO:journalctl@ceph.mon.b.smithi139.stdout: CephadmServe(self)._create_daemon(daemon_spec, reconfig=(action == 'reconfig'))) 2024-03-24T11:22:12.343 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/usr/share/ceph/mgr/cephadm/module.py", line 739, in wait_async 2024-03-24T11:22:12.343 INFO:journalctl@ceph.mon.b.smithi139.stdout: return self.event_loop.get_result(coro, timeout) 2024-03-24T11:22:12.343 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/usr/share/ceph/mgr/cephadm/ssh.py", line 64, in get_result 2024-03-24T11:22:12.343 INFO:journalctl@ceph.mon.b.smithi139.stdout: return future.result(timeout) 2024-03-24T11:22:12.344 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/lib64/python3.6/concurrent/futures/_base.py", line 432, in result 2024-03-24T11:22:12.344 INFO:journalctl@ceph.mon.b.smithi139.stdout: return self.__get_result() 2024-03-24T11:22:12.344 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/lib64/python3.6/concurrent/futures/_base.py", line 384, in __get_result 2024-03-24T11:22:12.344 INFO:journalctl@ceph.mon.b.smithi139.stdout: raise self._exception 2024-03-24T11:22:12.344 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1369, in _create_daemon 2024-03-24T11:22:12.344 INFO:journalctl@ceph.mon.b.smithi139.stdout: config_blobs=daemon_spec.final_config, 2024-03-24T11:22:12.344 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1671, in _run_cephadm 2024-03-24T11:22:12.344 INFO:journalctl@ceph.mon.b.smithi139.stdout: f'cephadm exited with an error code: {code}, stderr: {err}') 2024-03-24T11:22:12.344 INFO:journalctl@ceph.mon.b.smithi139.stdout: orchestrator._interface.OrchestratorError: cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi062.dggsvk ... 2024-03-24T11:22:12.344 INFO:journalctl@ceph.mon.b.smithi139.stdout: Creating ceph-iscsi config... 2024-03-24T11:22:12.344 INFO:journalctl@ceph.mon.b.smithi139.stdout: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/iscsi-gateway.cfg 2024-03-24T11:22:12.344 INFO:journalctl@ceph.mon.b.smithi139.stdout: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/tcmu-runner-entrypoint.sh 2024-03-24T11:22:12.344 INFO:journalctl@ceph.mon.b.smithi139.stdout: Non-zero exit code 1 from systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk 2024-03-24T11:22:12.345 INFO:journalctl@ceph.mon.b.smithi139.stdout: systemctl: stderr Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:22:12.345 INFO:journalctl@ceph.mon.b.smithi139.stdout: systemctl: stderr See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:22:12.345 INFO:journalctl@ceph.mon.b.smithi139.stdout: Traceback (most recent call last): 2024-03-24T11:22:12.345 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-24T11:22:12.345 INFO:journalctl@ceph.mon.b.smithi139.stdout: return _run_code(code, main_globals, None, 2024-03-24T11:22:12.345 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-24T11:22:12.345 INFO:journalctl@ceph.mon.b.smithi139.stdout: exec(code, run_globals) 2024-03-24T11:22:12.345 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5520, in 2024-03-24T11:22:12.345 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5508, in main 2024-03-24T11:22:12.345 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3038, in command_deploy_from 2024-03-24T11:22:12.345 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3073, in _common_deploy 2024-03-24T11:22:12.345 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3093, in _deploy_daemon_container 2024-03-24T11:22:12.345 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 1065, in deploy_daemon 2024-03-24T11:22:12.346 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-03-24T11:22:12.346 INFO:journalctl@ceph.mon.b.smithi139.stdout: RuntimeError: Failed command: systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk: Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:22:12.346 INFO:journalctl@ceph.mon.b.smithi139.stdout: See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:22:12.346 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:12 smithi139 ceph-mon[25719]: Reconfiguring prometheus.a (dependencies changed)... 2024-03-24T11:22:12.346 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:12 smithi139 ceph-mon[25719]: Reconfiguring daemon prometheus.a on smithi139 2024-03-24T11:22:13.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:13 smithi062 ceph-mon[24799]: pgmap v8: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2024-03-24T11:22:13.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:13 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:13.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:13 smithi062 ceph-mon[29666]: pgmap v8: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2024-03-24T11:22:13.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:13 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:13.343 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:13 smithi139 ceph-mon[25719]: pgmap v8: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2024-03-24T11:22:13.343 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:13 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:13.905 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:13 smithi139 systemd[1]: Stopping Ceph prometheus.a for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:22:13.905 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:13 smithi139 bash[60579]: Error: no container with name or ID "ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus.a" found: no such container 2024-03-24T11:22:14.190 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[55656]: ts=2024-03-24T11:22:13.905Z caller=main.go:775 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-03-24T11:22:14.191 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[55656]: ts=2024-03-24T11:22:13.905Z caller=main.go:798 level=info msg="Stopping scrape discovery manager..." 2024-03-24T11:22:14.191 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[55656]: ts=2024-03-24T11:22:13.905Z caller=main.go:812 level=info msg="Stopping notify discovery manager..." 2024-03-24T11:22:14.191 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[55656]: ts=2024-03-24T11:22:13.905Z caller=main.go:834 level=info msg="Stopping scrape manager..." 2024-03-24T11:22:14.191 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[55656]: ts=2024-03-24T11:22:13.905Z caller=main.go:794 level=info msg="Scrape discovery manager stopped" 2024-03-24T11:22:14.191 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[55656]: ts=2024-03-24T11:22:13.905Z caller=main.go:808 level=info msg="Notify discovery manager stopped" 2024-03-24T11:22:14.191 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[55656]: ts=2024-03-24T11:22:13.905Z caller=manager.go:945 level=info component="rule manager" msg="Stopping rule manager..." 2024-03-24T11:22:14.191 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[55656]: ts=2024-03-24T11:22:13.905Z caller=main.go:828 level=info msg="Scrape manager stopped" 2024-03-24T11:22:14.191 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[55656]: ts=2024-03-24T11:22:13.905Z caller=manager.go:955 level=info component="rule manager" msg="Rule manager stopped" 2024-03-24T11:22:14.191 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[55656]: ts=2024-03-24T11:22:13.906Z caller=notifier.go:600 level=info component=notifier msg="Stopping notification manager..." 2024-03-24T11:22:14.191 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[55656]: ts=2024-03-24T11:22:13.906Z caller=main.go:1054 level=info msg="Notifier manager stopped" 2024-03-24T11:22:14.191 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[55656]: ts=2024-03-24T11:22:13.906Z caller=main.go:1066 level=info msg="See you next time!" 2024-03-24T11:22:14.191 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:13 smithi139 podman[60590]: 2024-03-24 11:22:13.91541691 +0000 UTC m=+0.131517247 container died 5c0c71f4d276d17a5bbb8f08f35a21485368788f3a16842bc4346b323bfe6c33 (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-24T11:22:14.530 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:14 smithi139 podman[60590]: 2024-03-24 11:22:14.304861797 +0000 UTC m=+0.520962138 container cleanup 5c0c71f4d276d17a5bbb8f08f35a21485368788f3a16842bc4346b323bfe6c33 (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-24T11:22:14.531 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:14 smithi139 bash[60590]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a 2024-03-24T11:22:14.531 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:14 smithi139 bash[60616]: Error: no container with name or ID "ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus.a" found: no such container 2024-03-24T11:22:14.781 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:14 smithi139 podman[60605]: 2024-03-24 11:22:14.530994036 +0000 UTC m=+0.617109739 container remove 5c0c71f4d276d17a5bbb8f08f35a21485368788f3a16842bc4346b323bfe6c33 (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-24T11:22:14.781 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:14 smithi139 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@prometheus.a.service: Deactivated successfully. 2024-03-24T11:22:14.781 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:14 smithi139 systemd[1]: Stopped Ceph prometheus.a for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:22:14.782 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:14 smithi139 systemd[1]: Starting Ceph prometheus.a for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:22:14.782 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:14 smithi139 podman[60685]: 2024-03-24 11:22:14.669296723 +0000 UTC m=+0.018847302 image pull 514e6a882f6e74806a5856468489eeff8d7106095557578da96935e4d0ba4d9d quay.io/prometheus/prometheus:v2.33.4 2024-03-24T11:22:14.782 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:14 smithi139 podman[60685]: 2024-03-24 11:22:14.781890568 +0000 UTC m=+0.131441187 container create 2ed47f810d33a3b1e18ee293bfcbe12275d0c43227d2948938e31d807355fcb7 (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-24T11:22:15.084 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:14 smithi139 podman[60685]: 2024-03-24 11:22:14.992543424 +0000 UTC m=+0.342094001 container init 2ed47f810d33a3b1e18ee293bfcbe12275d0c43227d2948938e31d807355fcb7 (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-24T11:22:15.084 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:14 smithi139 podman[60685]: 2024-03-24 11:22:14.995165774 +0000 UTC m=+0.344716354 container start 2ed47f810d33a3b1e18ee293bfcbe12275d0c43227d2948938e31d807355fcb7 (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-24T11:22:15.084 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:15 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:22:15.019Z caller=main.go:475 level=info msg="No time or size retention was set so using the default time retention" duration=15d 2024-03-24T11:22:15.084 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:15 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:22:15.019Z caller=main.go:512 level=info msg="Starting Prometheus" version="(version=2.33.4, branch=HEAD, revision=83032011a5d3e6102624fe58241a374a7201fee8)" 2024-03-24T11:22:15.085 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:15 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:22:15.019Z caller=main.go:517 level=info build_context="(go=go1.17.7, user=root@d13bf69e7be8, date=20220222-16:51:28)" 2024-03-24T11:22:15.085 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:15 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:22:15.019Z caller=main.go:518 level=info host_details="(Linux 5.14.0-430.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024 x86_64 smithi139 (none))" 2024-03-24T11:22:15.085 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:15 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:22:15.019Z caller=main.go:519 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-03-24T11:22:15.085 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:15 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:22:15.019Z caller=main.go:520 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-03-24T11:22:15.085 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:15 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:22:15.020Z caller=web.go:570 level=info component=web msg="Start listening for connections" address=:9095 2024-03-24T11:22:15.085 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:15 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:22:15.021Z caller=main.go:923 level=info msg="Starting TSDB ..." 2024-03-24T11:22:15.085 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:15 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:22:15.022Z caller=tls_config.go:195 level=info component=web msg="TLS is disabled." http2=false 2024-03-24T11:22:15.085 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:15 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:22:15.023Z caller=head.go:493 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-03-24T11:22:15.085 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:15 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:22:15.023Z caller=head.go:527 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.371µs 2024-03-24T11:22:15.085 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:15 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:22:15.023Z caller=head.go:533 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-03-24T11:22:15.086 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:15 smithi139 bash[60685]: 2ed47f810d33a3b1e18ee293bfcbe12275d0c43227d2948938e31d807355fcb7 2024-03-24T11:22:15.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:15 smithi139 ceph-mon[25719]: pgmap v9: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-24T11:22:15.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:15 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:15.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:15 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:15.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:15 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T11:22:15.358 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:15 smithi139 systemd[1]: Started Ceph prometheus.a for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:22:15.358 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:15 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:22:15.225Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=5 2024-03-24T11:22:15.358 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:15 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:22:15.225Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=5 2024-03-24T11:22:15.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:15 smithi062 ceph-mon[29666]: pgmap v9: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-24T11:22:15.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:15 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:15.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:15 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:15.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:15 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T11:22:15.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:15 smithi062 ceph-mon[24799]: pgmap v9: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-24T11:22:15.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:15 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:15.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:15 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:15.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:15 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T11:22:15.748 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:15 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:22:15.386Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=2 maxSegment=5 2024-03-24T11:22:15.748 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:15 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:22:15.467Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=3 maxSegment=5 2024-03-24T11:22:15.748 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:15 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:22:15.512Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=4 maxSegment=5 2024-03-24T11:22:15.748 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:15 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:22:15.512Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=5 maxSegment=5 2024-03-24T11:22:15.748 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:15 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:22:15.512Z caller=head.go:610 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=39.456µs wal_replay_duration=489.054896ms total_replay_duration=489.106747ms 2024-03-24T11:22:15.748 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:15 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:22:15.517Z caller=main.go:944 level=info fs_type=XFS_SUPER_MAGIC 2024-03-24T11:22:15.748 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:15 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:22:15.517Z caller=main.go:947 level=info msg="TSDB started" 2024-03-24T11:22:15.748 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:15 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:22:15.517Z caller=main.go:1128 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-03-24T11:22:15.748 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:15 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:22:15.541Z caller=main.go:1165 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=23.729663ms db_storage=856ns remote_storage=1.688µs web_handler=381ns query_engine=821ns scrape=538.3µs scrape_sd=157.877µs notify=17.998µs notify_sd=20.039µs rules=22.531359ms 2024-03-24T11:22:15.748 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:15 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:22:15.541Z caller=main.go:896 level=info msg="Server is ready to receive web requests." 2024-03-24T11:22:16.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:16 smithi139 ceph-mon[25719]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T11:22:16.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:16 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:16.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:16 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:16.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:16 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:16.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:16 smithi139 ceph-mon[25719]: pgmap v10: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-24T11:22:16.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:16 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:16.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:16 smithi062 ceph-mon[29666]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T11:22:16.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:16 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:16.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:16 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:16.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:16 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:16.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:16 smithi062 ceph-mon[29666]: pgmap v10: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-24T11:22:16.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:16 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:16.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:16 smithi062 ceph-mon[24799]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T11:22:16.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:16 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:16.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:16 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:16.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:16 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:16.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:16 smithi062 ceph-mon[24799]: pgmap v10: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-24T11:22:16.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:16 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:17.441 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:17 smithi139 ceph-mon[25719]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-03-24T11:22:17.441 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T11:22:17.441 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T11:22:17.441 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:17.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:17 smithi062 ceph-mon[24799]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-03-24T11:22:17.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T11:22:17.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T11:22:17.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:17.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:17 smithi062 ceph-mon[29666]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-03-24T11:22:17.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T11:22:17.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T11:22:17.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:18.313 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:18 smithi139 ceph-mon[25719]: Upgrade: Updating mgr.x 2024-03-24T11:22:18.313 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:18 smithi139 ceph-mon[25719]: Deploying daemon mgr.x on smithi139 2024-03-24T11:22:18.314 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:18.314 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:22:18.314 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:18 smithi139 ceph-mon[25719]: pgmap v11: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-24T11:22:18.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:18 smithi062 ceph-mon[24799]: Upgrade: Updating mgr.x 2024-03-24T11:22:18.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:18 smithi062 ceph-mon[24799]: Deploying daemon mgr.x on smithi139 2024-03-24T11:22:18.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:18.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:22:18.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:18 smithi062 ceph-mon[24799]: pgmap v11: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-24T11:22:18.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:18 smithi062 ceph-mon[29666]: Upgrade: Updating mgr.x 2024-03-24T11:22:18.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:18 smithi062 ceph-mon[29666]: Deploying daemon mgr.x on smithi139 2024-03-24T11:22:18.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:18.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:22:18.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:18 smithi062 ceph-mon[29666]: pgmap v11: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-24T11:22:18.857 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:18 smithi139 systemd[1]: Stopping Ceph mgr.x for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:22:19.160 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:18 smithi139 podman[61028]: 2024-03-24 11:22:18.865573655 +0000 UTC m=+0.127506984 container died ed2c6bf552c087e96f16fd2133789a4845418fa6f0dc149f47a7cc5e0b8e5b50 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-24T11:22:19.443 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:19 smithi139 ceph-mon[25719]: pgmap v12: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-24T11:22:19.444 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:19 smithi139 podman[61028]: 2024-03-24 11:22:19.242841744 +0000 UTC m=+0.504775076 container cleanup ed2c6bf552c087e96f16fd2133789a4845418fa6f0dc149f47a7cc5e0b8e5b50 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x, RELEASE=squid-db0330b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-24T11:22:19.444 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:19 smithi139 bash[61028]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x 2024-03-24T11:22:19.715 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:19 smithi139 podman[61042]: 2024-03-24 11:22:19.519396335 +0000 UTC m=+0.656173793 container remove ed2c6bf552c087e96f16fd2133789a4845418fa6f0dc149f47a7cc5e0b8e5b50 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-24T11:22:19.715 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:19 smithi139 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mgr.x.service: Main process exited, code=exited, status=143/n/a 2024-03-24T11:22:19.715 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:19 smithi139 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mgr.x.service: Failed with result 'exit-code'. 2024-03-24T11:22:19.715 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:19 smithi139 systemd[1]: Stopped Ceph mgr.x for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:22:19.715 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:19 smithi139 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mgr.x.service: Consumed 30.142s CPU time. 2024-03-24T11:22:19.726 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:19 smithi062 ceph-mon[24799]: pgmap v12: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-24T11:22:19.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:19 smithi062 ceph-mon[29666]: pgmap v12: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-24T11:22:20.030 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:19 smithi139 systemd[1]: Starting Ceph mgr.x for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:22:20.314 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:20 smithi139 podman[61177]: 2024-03-24 11:22:19.933322431 +0000 UTC m=+0.018270467 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:22:20.315 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:20 smithi139 podman[61177]: 2024-03-24 11:22:20.037417691 +0000 UTC m=+0.122365720 container create 416473ec6776de2084e44695044ffc9d600d44019d24ef77e5725b3592bd5c02 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T11:22:20.315 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:20 smithi139 podman[61177]: 2024-03-24 11:22:20.239803977 +0000 UTC m=+0.324752003 container init 416473ec6776de2084e44695044ffc9d600d44019d24ef77e5725b3592bd5c02 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-db0330b, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-24T11:22:20.315 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:20 smithi139 podman[61177]: 2024-03-24 11:22:20.242885615 +0000 UTC m=+0.327833644 container start 416473ec6776de2084e44695044ffc9d600d44019d24ef77e5725b3592bd5c02 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-db0330b, ceph=True, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-03-24T11:22:20.315 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:20 smithi139 ceph-mgr[61196]: -- 172.21.15.139:0/1611000524 <== mon.2 v2:172.21.15.139:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55848b55a1a0 con 0x55848b531000 2024-03-24T11:22:20.315 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:20 smithi139 bash[61177]: 416473ec6776de2084e44695044ffc9d600d44019d24ef77e5725b3592bd5c02 2024-03-24T11:22:20.607 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:20 smithi139 systemd[1]: Started Ceph mgr.x for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:22:21.108 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:20 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[61192]: 2024-03-24T11:22:20.791+0000 7fbb30ba5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-24T11:22:21.108 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:20 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[61192]: 2024-03-24T11:22:20.885+0000 7fbb30ba5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-24T11:22:21.158 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:20 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:22:20] "GET /metrics HTTP/1.1" 200 38315 "" "Prometheus/2.33.4" 2024-03-24T11:22:21.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:21.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:21.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:21.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:21.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:21 smithi062 ceph-mon[24799]: pgmap v13: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 198 B/s rd, 0 op/s 2024-03-24T11:22:21.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:21 smithi062 ceph-mon[24799]: pgmap v14: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 246 B/s rd, 0 op/s 2024-03-24T11:22:21.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:21.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:21 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:21.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:21 smithi062 ceph-mon[24799]: Reconfiguring iscsi.foo.smithi062.dggsvk (dependencies changed)... 2024-03-24T11:22:21.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi062.dggsvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-03-24T11:22:21.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:21.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:21 smithi062 ceph-mon[24799]: Reconfiguring daemon iscsi.foo.smithi062.dggsvk on smithi062 2024-03-24T11:22:21.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:21.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:21.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:21.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:21.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:21 smithi062 ceph-mon[29666]: pgmap v13: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 198 B/s rd, 0 op/s 2024-03-24T11:22:21.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:21 smithi062 ceph-mon[29666]: pgmap v14: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 246 B/s rd, 0 op/s 2024-03-24T11:22:21.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:21.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:21 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:21.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:21 smithi062 ceph-mon[29666]: Reconfiguring iscsi.foo.smithi062.dggsvk (dependencies changed)... 2024-03-24T11:22:21.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi062.dggsvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-03-24T11:22:21.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:21.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:21 smithi062 ceph-mon[29666]: Reconfiguring daemon iscsi.foo.smithi062.dggsvk on smithi062 2024-03-24T11:22:21.768 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:21 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[61192]: 2024-03-24T11:22:21.532+0000 7fbb30ba5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-24T11:22:21.768 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:21 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[61192]: 2024-03-24T11:22:21.626+0000 7fbb30ba5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-24T11:22:21.768 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:21.768 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:21.768 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:21.768 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:21.768 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:21 smithi139 ceph-mon[25719]: pgmap v13: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 198 B/s rd, 0 op/s 2024-03-24T11:22:21.768 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:21 smithi139 ceph-mon[25719]: pgmap v14: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 246 B/s rd, 0 op/s 2024-03-24T11:22:21.768 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:21.768 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:21 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:21.768 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:21 smithi139 ceph-mon[25719]: Reconfiguring iscsi.foo.smithi062.dggsvk (dependencies changed)... 2024-03-24T11:22:21.769 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi062.dggsvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-03-24T11:22:21.769 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:21.769 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:21 smithi139 ceph-mon[25719]: Reconfiguring daemon iscsi.foo.smithi062.dggsvk on smithi062 2024-03-24T11:22:22.107 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:21 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[61192]: 2024-03-24T11:22:21.767+0000 7fbb30ba5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-24T11:22:22.108 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:21 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[61192]: 2024-03-24T11:22:21.864+0000 7fbb30ba5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-24T11:22:22.108 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:21 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[61192]: 2024-03-24T11:22:21.947+0000 7fbb30ba5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-24T11:22:22.507 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:22 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/3355062327' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-24T11:22:22.507 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:22 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[61192]: 2024-03-24T11:22:22.214+0000 7fbb30ba5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-24T11:22:22.507 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:22 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[61192]: 2024-03-24T11:22:22.324+0000 7fbb30ba5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-24T11:22:22.507 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:22 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[61192]: 2024-03-24T11:22:22.411+0000 7fbb30ba5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-24T11:22:22.507 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:22 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[61192]: 2024-03-24T11:22:22.506+0000 7fbb30ba5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-24T11:22:22.519 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:22 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/3355062327' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-24T11:22:22.520 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:22 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/3355062327' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-24T11:22:22.857 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:22 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[61192]: 2024-03-24T11:22:22.685+0000 7fbb30ba5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-24T11:22:23.936 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:23 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[61192]: 2024-03-24T11:22:23.663+0000 7fbb30ba5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-24T11:22:23.936 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:23 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[61192]: 2024-03-24T11:22:23.795+0000 7fbb30ba5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-24T11:22:24.117 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:23 smithi062 ceph-mon[24799]: pgmap v15: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:22:24.118 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:23 smithi062 ceph-mon[29666]: pgmap v15: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:22:24.239 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:24 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[61192]: 2024-03-24T11:22:24.140+0000 7fbb30ba5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-24T11:22:24.239 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:23 smithi139 ceph-mon[25719]: pgmap v15: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:22:24.477 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:22:24.116+0000 7f95852cc700 -1 log_channel(cephadm) log [ERR] : cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi062.dggsvk ... 2024-03-24T11:22:24.477 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: Creating ceph-iscsi config... 2024-03-24T11:22:24.477 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/iscsi-gateway.cfg 2024-03-24T11:22:24.477 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/tcmu-runner-entrypoint.sh 2024-03-24T11:22:24.477 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: Non-zero exit code 1 from systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk 2024-03-24T11:22:24.477 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: systemctl: stderr Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:22:24.477 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: systemctl: stderr See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:22:24.478 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: Traceback (most recent call last): 2024-03-24T11:22:24.478 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-24T11:22:24.478 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: return _run_code(code, main_globals, None, 2024-03-24T11:22:24.478 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-24T11:22:24.478 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: exec(code, run_globals) 2024-03-24T11:22:24.478 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5520, in 2024-03-24T11:22:24.478 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5508, in main 2024-03-24T11:22:24.478 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3038, in command_deploy_from 2024-03-24T11:22:24.478 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3073, in _common_deploy 2024-03-24T11:22:24.478 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3093, in _deploy_daemon_container 2024-03-24T11:22:24.478 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 1065, in deploy_daemon 2024-03-24T11:22:24.478 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-03-24T11:22:24.479 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: RuntimeError: Failed command: systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk: Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:22:24.479 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:22:24.479 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: Traceback (most recent call last): 2024-03-24T11:22:24.479 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1109, in _check_daemons 2024-03-24T11:22:24.479 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: self.mgr._daemon_action(daemon_spec, action=action) 2024-03-24T11:22:24.479 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/usr/share/ceph/mgr/cephadm/module.py", line 2372, in _daemon_action 2024-03-24T11:22:24.479 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: CephadmServe(self)._create_daemon(daemon_spec, reconfig=(action == 'reconfig'))) 2024-03-24T11:22:24.479 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/usr/share/ceph/mgr/cephadm/module.py", line 739, in wait_async 2024-03-24T11:22:24.480 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: return self.event_loop.get_result(coro, timeout) 2024-03-24T11:22:24.480 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/usr/share/ceph/mgr/cephadm/ssh.py", line 64, in get_result 2024-03-24T11:22:24.480 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: return future.result(timeout) 2024-03-24T11:22:24.480 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/lib64/python3.6/concurrent/futures/_base.py", line 432, in result 2024-03-24T11:22:24.480 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: return self.__get_result() 2024-03-24T11:22:24.480 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/lib64/python3.6/concurrent/futures/_base.py", line 384, in __get_result 2024-03-24T11:22:24.480 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: raise self._exception 2024-03-24T11:22:24.480 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1369, in _create_daemon 2024-03-24T11:22:24.480 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: config_blobs=daemon_spec.final_config, 2024-03-24T11:22:24.480 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1671, in _run_cephadm 2024-03-24T11:22:24.480 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: f'cephadm exited with an error code: {code}, stderr: {err}') 2024-03-24T11:22:24.480 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: orchestrator._interface.OrchestratorError: cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi062.dggsvk ... 2024-03-24T11:22:24.480 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: Creating ceph-iscsi config... 2024-03-24T11:22:24.480 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/iscsi-gateway.cfg 2024-03-24T11:22:24.481 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/tcmu-runner-entrypoint.sh 2024-03-24T11:22:24.481 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: Non-zero exit code 1 from systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk 2024-03-24T11:22:24.481 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: systemctl: stderr Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:22:24.481 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: systemctl: stderr See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:22:24.481 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: Traceback (most recent call last): 2024-03-24T11:22:24.481 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-24T11:22:24.481 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: return _run_code(code, main_globals, None, 2024-03-24T11:22:24.481 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-24T11:22:24.481 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: exec(code, run_globals) 2024-03-24T11:22:24.481 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5520, in 2024-03-24T11:22:24.481 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5508, in main 2024-03-24T11:22:24.481 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3038, in command_deploy_from 2024-03-24T11:22:24.481 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3073, in _common_deploy 2024-03-24T11:22:24.481 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3093, in _deploy_daemon_container 2024-03-24T11:22:24.482 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 1065, in deploy_daemon 2024-03-24T11:22:24.482 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-03-24T11:22:24.482 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: RuntimeError: Failed command: systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk: Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:22:24.482 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:22:24.608 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:24 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[61192]: 2024-03-24T11:22:24.239+0000 7fbb30ba5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-24T11:22:24.608 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:24 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[61192]: 2024-03-24T11:22:24.437+0000 7fbb30ba5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-24T11:22:24.934 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:24 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[61192]: 2024-03-24T11:22:24.626+0000 7fbb30ba5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-24T11:22:25.101 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-mon[24799]: cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi062.dggsvk ... 2024-03-24T11:22:25.102 INFO:journalctl@ceph.mon.a.smithi062.stdout: Creating ceph-iscsi config... 2024-03-24T11:22:25.102 INFO:journalctl@ceph.mon.a.smithi062.stdout: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/iscsi-gateway.cfg 2024-03-24T11:22:25.102 INFO:journalctl@ceph.mon.a.smithi062.stdout: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/tcmu-runner-entrypoint.sh 2024-03-24T11:22:25.102 INFO:journalctl@ceph.mon.a.smithi062.stdout: Non-zero exit code 1 from systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk 2024-03-24T11:22:25.102 INFO:journalctl@ceph.mon.a.smithi062.stdout: systemctl: stderr Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:22:25.102 INFO:journalctl@ceph.mon.a.smithi062.stdout: systemctl: stderr See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:22:25.102 INFO:journalctl@ceph.mon.a.smithi062.stdout: Traceback (most recent call last): 2024-03-24T11:22:25.102 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-24T11:22:25.102 INFO:journalctl@ceph.mon.a.smithi062.stdout: return _run_code(code, main_globals, None, 2024-03-24T11:22:25.102 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-24T11:22:25.102 INFO:journalctl@ceph.mon.a.smithi062.stdout: exec(code, run_globals) 2024-03-24T11:22:25.102 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5520, in 2024-03-24T11:22:25.102 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5508, in main 2024-03-24T11:22:25.103 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3038, in command_deploy_from 2024-03-24T11:22:25.103 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3073, in _common_deploy 2024-03-24T11:22:25.103 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3093, in _deploy_daemon_container 2024-03-24T11:22:25.103 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 1065, in deploy_daemon 2024-03-24T11:22:25.103 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-03-24T11:22:25.103 INFO:journalctl@ceph.mon.a.smithi062.stdout: RuntimeError: Failed command: systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk: Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:22:25.103 INFO:journalctl@ceph.mon.a.smithi062.stdout: See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:22:25.103 INFO:journalctl@ceph.mon.a.smithi062.stdout: Traceback (most recent call last): 2024-03-24T11:22:25.103 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1109, in _check_daemons 2024-03-24T11:22:25.103 INFO:journalctl@ceph.mon.a.smithi062.stdout: self.mgr._daemon_action(daemon_spec, action=action) 2024-03-24T11:22:25.103 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/usr/share/ceph/mgr/cephadm/module.py", line 2372, in _daemon_action 2024-03-24T11:22:25.103 INFO:journalctl@ceph.mon.a.smithi062.stdout: CephadmServe(self)._create_daemon(daemon_spec, reconfig=(action == 'reconfig'))) 2024-03-24T11:22:25.104 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/usr/share/ceph/mgr/cephadm/module.py", line 739, in wait_async 2024-03-24T11:22:25.104 INFO:journalctl@ceph.mon.a.smithi062.stdout: return self.event_loop.get_result(coro, timeout) 2024-03-24T11:22:25.104 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/usr/share/ceph/mgr/cephadm/ssh.py", line 64, in get_result 2024-03-24T11:22:25.104 INFO:journalctl@ceph.mon.a.smithi062.stdout: return future.result(timeout) 2024-03-24T11:22:25.104 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib64/python3.6/concurrent/futures/_base.py", line 432, in result 2024-03-24T11:22:25.104 INFO:journalctl@ceph.mon.a.smithi062.stdout: return self.__get_result() 2024-03-24T11:22:25.104 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib64/python3.6/concurrent/futures/_base.py", line 384, in __get_result 2024-03-24T11:22:25.104 INFO:journalctl@ceph.mon.a.smithi062.stdout: raise self._exception 2024-03-24T11:22:25.104 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1369, in _create_daemon 2024-03-24T11:22:25.105 INFO:journalctl@ceph.mon.a.smithi062.stdout: config_blobs=daemon_spec.final_config, 2024-03-24T11:22:25.105 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1671, in _run_cephadm 2024-03-24T11:22:25.105 INFO:journalctl@ceph.mon.a.smithi062.stdout: f'cephadm exited with an error code: {code}, stderr: {err}') 2024-03-24T11:22:25.105 INFO:journalctl@ceph.mon.a.smithi062.stdout: orchestrator._interface.OrchestratorError: cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi062.dggsvk ... 2024-03-24T11:22:25.105 INFO:journalctl@ceph.mon.a.smithi062.stdout: Creating ceph-iscsi config... 2024-03-24T11:22:25.105 INFO:journalctl@ceph.mon.a.smithi062.stdout: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/iscsi-gateway.cfg 2024-03-24T11:22:25.105 INFO:journalctl@ceph.mon.a.smithi062.stdout: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/tcmu-runner-entrypoint.sh 2024-03-24T11:22:25.105 INFO:journalctl@ceph.mon.a.smithi062.stdout: Non-zero exit code 1 from systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk 2024-03-24T11:22:25.106 INFO:journalctl@ceph.mon.a.smithi062.stdout: systemctl: stderr Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:22:25.106 INFO:journalctl@ceph.mon.a.smithi062.stdout: systemctl: stderr See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:22:25.106 INFO:journalctl@ceph.mon.a.smithi062.stdout: Traceback (most recent call last): 2024-03-24T11:22:25.106 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-24T11:22:25.106 INFO:journalctl@ceph.mon.a.smithi062.stdout: return _run_code(code, main_globals, None, 2024-03-24T11:22:25.106 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-24T11:22:25.107 INFO:journalctl@ceph.mon.a.smithi062.stdout: exec(code, run_globals) 2024-03-24T11:22:25.107 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5520, in 2024-03-24T11:22:25.107 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5508, in main 2024-03-24T11:22:25.107 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3038, in command_deploy_from 2024-03-24T11:22:25.107 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3073, in _common_deploy 2024-03-24T11:22:25.108 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-mon[29666]: cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi062.dggsvk ... 2024-03-24T11:22:25.108 INFO:journalctl@ceph.mon.c.smithi062.stdout: Creating ceph-iscsi config... 2024-03-24T11:22:25.108 INFO:journalctl@ceph.mon.c.smithi062.stdout: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/iscsi-gateway.cfg 2024-03-24T11:22:25.108 INFO:journalctl@ceph.mon.c.smithi062.stdout: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/tcmu-runner-entrypoint.sh 2024-03-24T11:22:25.108 INFO:journalctl@ceph.mon.c.smithi062.stdout: Non-zero exit code 1 from systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk 2024-03-24T11:22:25.108 INFO:journalctl@ceph.mon.c.smithi062.stdout: systemctl: stderr Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:22:25.108 INFO:journalctl@ceph.mon.c.smithi062.stdout: systemctl: stderr See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:22:25.109 INFO:journalctl@ceph.mon.c.smithi062.stdout: Traceback (most recent call last): 2024-03-24T11:22:25.109 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-24T11:22:25.109 INFO:journalctl@ceph.mon.c.smithi062.stdout: return _run_code(code, main_globals, None, 2024-03-24T11:22:25.109 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-24T11:22:25.109 INFO:journalctl@ceph.mon.c.smithi062.stdout: exec(code, run_globals) 2024-03-24T11:22:25.109 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5520, in 2024-03-24T11:22:25.109 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5508, in main 2024-03-24T11:22:25.109 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3038, in command_deploy_from 2024-03-24T11:22:25.109 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3073, in _common_deploy 2024-03-24T11:22:25.109 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3093, in _deploy_daemon_container 2024-03-24T11:22:25.110 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 1065, in deploy_daemon 2024-03-24T11:22:25.110 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-03-24T11:22:25.110 INFO:journalctl@ceph.mon.c.smithi062.stdout: RuntimeError: Failed command: systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk: Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:22:25.110 INFO:journalctl@ceph.mon.c.smithi062.stdout: See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:22:25.110 INFO:journalctl@ceph.mon.c.smithi062.stdout: Traceback (most recent call last): 2024-03-24T11:22:25.110 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1109, in _check_daemons 2024-03-24T11:22:25.110 INFO:journalctl@ceph.mon.c.smithi062.stdout: self.mgr._daemon_action(daemon_spec, action=action) 2024-03-24T11:22:25.110 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/usr/share/ceph/mgr/cephadm/module.py", line 2372, in _daemon_action 2024-03-24T11:22:25.110 INFO:journalctl@ceph.mon.c.smithi062.stdout: CephadmServe(self)._create_daemon(daemon_spec, reconfig=(action == 'reconfig'))) 2024-03-24T11:22:25.110 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/usr/share/ceph/mgr/cephadm/module.py", line 739, in wait_async 2024-03-24T11:22:25.110 INFO:journalctl@ceph.mon.c.smithi062.stdout: return self.event_loop.get_result(coro, timeout) 2024-03-24T11:22:25.111 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/usr/share/ceph/mgr/cephadm/ssh.py", line 64, in get_result 2024-03-24T11:22:25.111 INFO:journalctl@ceph.mon.c.smithi062.stdout: return future.result(timeout) 2024-03-24T11:22:25.111 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/lib64/python3.6/concurrent/futures/_base.py", line 432, in result 2024-03-24T11:22:25.111 INFO:journalctl@ceph.mon.c.smithi062.stdout: return self.__get_result() 2024-03-24T11:22:25.111 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/lib64/python3.6/concurrent/futures/_base.py", line 384, in __get_result 2024-03-24T11:22:25.111 INFO:journalctl@ceph.mon.c.smithi062.stdout: raise self._exception 2024-03-24T11:22:25.111 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1369, in _create_daemon 2024-03-24T11:22:25.111 INFO:journalctl@ceph.mon.c.smithi062.stdout: config_blobs=daemon_spec.final_config, 2024-03-24T11:22:25.111 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1671, in _run_cephadm 2024-03-24T11:22:25.111 INFO:journalctl@ceph.mon.c.smithi062.stdout: f'cephadm exited with an error code: {code}, stderr: {err}') 2024-03-24T11:22:25.111 INFO:journalctl@ceph.mon.c.smithi062.stdout: orchestrator._interface.OrchestratorError: cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi062.dggsvk ... 2024-03-24T11:22:25.111 INFO:journalctl@ceph.mon.c.smithi062.stdout: Creating ceph-iscsi config... 2024-03-24T11:22:25.111 INFO:journalctl@ceph.mon.c.smithi062.stdout: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/iscsi-gateway.cfg 2024-03-24T11:22:25.111 INFO:journalctl@ceph.mon.c.smithi062.stdout: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/tcmu-runner-entrypoint.sh 2024-03-24T11:22:25.112 INFO:journalctl@ceph.mon.c.smithi062.stdout: Non-zero exit code 1 from systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk 2024-03-24T11:22:25.112 INFO:journalctl@ceph.mon.c.smithi062.stdout: systemctl: stderr Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:22:25.112 INFO:journalctl@ceph.mon.c.smithi062.stdout: systemctl: stderr See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:22:25.112 INFO:journalctl@ceph.mon.c.smithi062.stdout: Traceback (most recent call last): 2024-03-24T11:22:25.112 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-24T11:22:25.112 INFO:journalctl@ceph.mon.c.smithi062.stdout: return _run_code(code, main_globals, None, 2024-03-24T11:22:25.112 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-24T11:22:25.112 INFO:journalctl@ceph.mon.c.smithi062.stdout: exec(code, run_globals) 2024-03-24T11:22:25.112 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5520, in 2024-03-24T11:22:25.112 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5508, in main 2024-03-24T11:22:25.112 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3038, in command_deploy_from 2024-03-24T11:22:25.112 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3073, in _common_deploy 2024-03-24T11:22:25.113 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3093, in _deploy_daemon_container 2024-03-24T11:22:25.113 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 1065, in deploy_daemon 2024-03-24T11:22:25.113 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-03-24T11:22:25.113 INFO:journalctl@ceph.mon.a.smithi062.stdout: RuntimeError: Failed command: systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk: Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:22:25.113 INFO:journalctl@ceph.mon.a.smithi062.stdout: See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:22:25.113 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:25.113 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:25.113 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-mon[24799]: Upgrade: Setting container_image for all mgr 2024-03-24T11:22:25.113 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:25.113 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]: dispatch 2024-03-24T11:22:25.113 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]': finished 2024-03-24T11:22:25.113 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]: dispatch 2024-03-24T11:22:25.114 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]': finished 2024-03-24T11:22:25.114 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:25.114 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:25.114 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-mon[24799]: pgmap v16: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 342 B/s rd, 0 op/s 2024-03-24T11:22:25.114 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-mon[24799]: pgmap v17: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 441 B/s rd, 0 op/s 2024-03-24T11:22:25.114 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:25.114 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:25.114 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-mon[24799]: Reconfiguring iscsi.foo.smithi062.dggsvk (dependencies changed)... 2024-03-24T11:22:25.114 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi062.dggsvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-03-24T11:22:25.115 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:25.115 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-mon[24799]: Reconfiguring daemon iscsi.foo.smithi062.dggsvk on smithi062 2024-03-24T11:22:25.115 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3093, in _deploy_daemon_container 2024-03-24T11:22:25.115 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 1065, in deploy_daemon 2024-03-24T11:22:25.115 INFO:journalctl@ceph.mon.c.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-03-24T11:22:25.115 INFO:journalctl@ceph.mon.c.smithi062.stdout: RuntimeError: Failed command: systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk: Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:22:25.115 INFO:journalctl@ceph.mon.c.smithi062.stdout: See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:22:25.115 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:25.115 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:25.115 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-mon[29666]: Upgrade: Setting container_image for all mgr 2024-03-24T11:22:25.115 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:25.115 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]: dispatch 2024-03-24T11:22:25.115 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]': finished 2024-03-24T11:22:25.116 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]: dispatch 2024-03-24T11:22:25.116 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]': finished 2024-03-24T11:22:25.116 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:25.116 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:25.116 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-mon[29666]: pgmap v16: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 342 B/s rd, 0 op/s 2024-03-24T11:22:25.116 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-mon[29666]: pgmap v17: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 441 B/s rd, 0 op/s 2024-03-24T11:22:25.116 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:25.116 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:25.116 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-mon[29666]: Reconfiguring iscsi.foo.smithi062.dggsvk (dependencies changed)... 2024-03-24T11:22:25.116 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi062.dggsvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-03-24T11:22:25.116 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:25.116 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:24 smithi062 ceph-mon[29666]: Reconfiguring daemon iscsi.foo.smithi062.dggsvk on smithi062 2024-03-24T11:22:25.232 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:24 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[61192]: 2024-03-24T11:22:24.933+0000 7fbb30ba5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-24T11:22:25.233 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:24 smithi139 ceph-mon[25719]: cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi062.dggsvk ... 2024-03-24T11:22:25.233 INFO:journalctl@ceph.mon.b.smithi139.stdout: Creating ceph-iscsi config... 2024-03-24T11:22:25.233 INFO:journalctl@ceph.mon.b.smithi139.stdout: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/iscsi-gateway.cfg 2024-03-24T11:22:25.233 INFO:journalctl@ceph.mon.b.smithi139.stdout: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/tcmu-runner-entrypoint.sh 2024-03-24T11:22:25.233 INFO:journalctl@ceph.mon.b.smithi139.stdout: Non-zero exit code 1 from systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk 2024-03-24T11:22:25.234 INFO:journalctl@ceph.mon.b.smithi139.stdout: systemctl: stderr Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:22:25.234 INFO:journalctl@ceph.mon.b.smithi139.stdout: systemctl: stderr See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:22:25.234 INFO:journalctl@ceph.mon.b.smithi139.stdout: Traceback (most recent call last): 2024-03-24T11:22:25.234 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-24T11:22:25.234 INFO:journalctl@ceph.mon.b.smithi139.stdout: return _run_code(code, main_globals, None, 2024-03-24T11:22:25.234 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-24T11:22:25.235 INFO:journalctl@ceph.mon.b.smithi139.stdout: exec(code, run_globals) 2024-03-24T11:22:25.235 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5520, in 2024-03-24T11:22:25.235 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5508, in main 2024-03-24T11:22:25.235 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3038, in command_deploy_from 2024-03-24T11:22:25.235 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3073, in _common_deploy 2024-03-24T11:22:25.235 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3093, in _deploy_daemon_container 2024-03-24T11:22:25.236 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 1065, in deploy_daemon 2024-03-24T11:22:25.236 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-03-24T11:22:25.236 INFO:journalctl@ceph.mon.b.smithi139.stdout: RuntimeError: Failed command: systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk: Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:22:25.236 INFO:journalctl@ceph.mon.b.smithi139.stdout: See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:22:25.236 INFO:journalctl@ceph.mon.b.smithi139.stdout: Traceback (most recent call last): 2024-03-24T11:22:25.236 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1109, in _check_daemons 2024-03-24T11:22:25.237 INFO:journalctl@ceph.mon.b.smithi139.stdout: self.mgr._daemon_action(daemon_spec, action=action) 2024-03-24T11:22:25.237 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/usr/share/ceph/mgr/cephadm/module.py", line 2372, in _daemon_action 2024-03-24T11:22:25.237 INFO:journalctl@ceph.mon.b.smithi139.stdout: CephadmServe(self)._create_daemon(daemon_spec, reconfig=(action == 'reconfig'))) 2024-03-24T11:22:25.237 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/usr/share/ceph/mgr/cephadm/module.py", line 739, in wait_async 2024-03-24T11:22:25.237 INFO:journalctl@ceph.mon.b.smithi139.stdout: return self.event_loop.get_result(coro, timeout) 2024-03-24T11:22:25.237 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/usr/share/ceph/mgr/cephadm/ssh.py", line 64, in get_result 2024-03-24T11:22:25.238 INFO:journalctl@ceph.mon.b.smithi139.stdout: return future.result(timeout) 2024-03-24T11:22:25.238 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/lib64/python3.6/concurrent/futures/_base.py", line 432, in result 2024-03-24T11:22:25.238 INFO:journalctl@ceph.mon.b.smithi139.stdout: return self.__get_result() 2024-03-24T11:22:25.238 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/lib64/python3.6/concurrent/futures/_base.py", line 384, in __get_result 2024-03-24T11:22:25.238 INFO:journalctl@ceph.mon.b.smithi139.stdout: raise self._exception 2024-03-24T11:22:25.239 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1369, in _create_daemon 2024-03-24T11:22:25.239 INFO:journalctl@ceph.mon.b.smithi139.stdout: config_blobs=daemon_spec.final_config, 2024-03-24T11:22:25.239 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1671, in _run_cephadm 2024-03-24T11:22:25.239 INFO:journalctl@ceph.mon.b.smithi139.stdout: f'cephadm exited with an error code: {code}, stderr: {err}') 2024-03-24T11:22:25.239 INFO:journalctl@ceph.mon.b.smithi139.stdout: orchestrator._interface.OrchestratorError: cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi062.dggsvk ... 2024-03-24T11:22:25.239 INFO:journalctl@ceph.mon.b.smithi139.stdout: Creating ceph-iscsi config... 2024-03-24T11:22:25.239 INFO:journalctl@ceph.mon.b.smithi139.stdout: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/iscsi-gateway.cfg 2024-03-24T11:22:25.240 INFO:journalctl@ceph.mon.b.smithi139.stdout: Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/tcmu-runner-entrypoint.sh 2024-03-24T11:22:25.240 INFO:journalctl@ceph.mon.b.smithi139.stdout: Non-zero exit code 1 from systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk 2024-03-24T11:22:25.240 INFO:journalctl@ceph.mon.b.smithi139.stdout: systemctl: stderr Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:22:25.240 INFO:journalctl@ceph.mon.b.smithi139.stdout: systemctl: stderr See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:22:25.240 INFO:journalctl@ceph.mon.b.smithi139.stdout: Traceback (most recent call last): 2024-03-24T11:22:25.241 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-24T11:22:25.241 INFO:journalctl@ceph.mon.b.smithi139.stdout: return _run_code(code, main_globals, None, 2024-03-24T11:22:25.241 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-24T11:22:25.241 INFO:journalctl@ceph.mon.b.smithi139.stdout: exec(code, run_globals) 2024-03-24T11:22:25.241 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5520, in 2024-03-24T11:22:25.241 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5508, in main 2024-03-24T11:22:25.241 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3038, in command_deploy_from 2024-03-24T11:22:25.242 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3073, in _common_deploy 2024-03-24T11:22:25.242 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3093, in _deploy_daemon_container 2024-03-24T11:22:25.242 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 1065, in deploy_daemon 2024-03-24T11:22:25.242 INFO:journalctl@ceph.mon.b.smithi139.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-03-24T11:22:25.242 INFO:journalctl@ceph.mon.b.smithi139.stdout: RuntimeError: Failed command: systemctl restart ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk: Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:22:25.242 INFO:journalctl@ceph.mon.b.smithi139.stdout: See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:22:25.242 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:24 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:25.243 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:24 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:25.243 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:24 smithi139 ceph-mon[25719]: Upgrade: Setting container_image for all mgr 2024-03-24T11:22:25.243 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:24 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:25.243 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:24 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]: dispatch 2024-03-24T11:22:25.243 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:24 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]': finished 2024-03-24T11:22:25.243 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:24 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]: dispatch 2024-03-24T11:22:25.243 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:24 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]': finished 2024-03-24T11:22:25.243 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:24 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:25.243 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:24 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:25.244 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:24 smithi139 ceph-mon[25719]: pgmap v16: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 342 B/s rd, 0 op/s 2024-03-24T11:22:25.244 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:24 smithi139 ceph-mon[25719]: pgmap v17: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 441 B/s rd, 0 op/s 2024-03-24T11:22:25.244 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:24 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:25.244 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:24 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:25.244 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:24 smithi139 ceph-mon[25719]: Reconfiguring iscsi.foo.smithi062.dggsvk (dependencies changed)... 2024-03-24T11:22:25.244 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:24 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi062.dggsvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-03-24T11:22:25.244 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:24 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:25.244 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:24 smithi139 ceph-mon[25719]: Reconfiguring daemon iscsi.foo.smithi062.dggsvk on smithi062 2024-03-24T11:22:25.489 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[61192]: 2024-03-24T11:22:25.231+0000 7fbb30ba5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-24T11:22:25.489 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[61192]: 2024-03-24T11:22:25.488+0000 7fbb30ba5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-24T11:22:26.396 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:26 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[61192]: 2024-03-24T11:22:26.353+0000 7fbb30ba5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-24T11:22:27.358 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:27 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[61192]: 2024-03-24T11:22:27.027+0000 7fbb30ba5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-24T11:22:27.358 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:27 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[61192]: 2024-03-24T11:22:27.134+0000 7fbb30ba5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-24T11:22:27.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:27 smithi139 ceph-mon[25719]: pgmap v18: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 441 B/s rd, 0 op/s 2024-03-24T11:22:27.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:27 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:27.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:27 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:27.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:27 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-03-24T11:22:27.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:27 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-03-24T11:22:27.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:27 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:27.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:27 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:27.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:27 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:27.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:27 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:27.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:27 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:27.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:27 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:27.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:27 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:27.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:27 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:27.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:27 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:27.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:27 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:27.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:27 smithi062 ceph-mon[24799]: pgmap v18: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 441 B/s rd, 0 op/s 2024-03-24T11:22:27.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:27 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:27.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:27 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:27.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:27 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-03-24T11:22:27.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:27 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-03-24T11:22:27.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:27 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:27.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:27 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:27.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:27 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:27.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:27 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:27.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:27 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:27.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:27 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:27.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:27 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:27.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:27 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:27.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:27 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:27.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:27 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:27.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:27 smithi062 ceph-mon[29666]: pgmap v18: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 441 B/s rd, 0 op/s 2024-03-24T11:22:27.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:27 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:27.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:27 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:27.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:27 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-03-24T11:22:27.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:27 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-03-24T11:22:27.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:27 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:27.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:27 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:27.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:27 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:27.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:27 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:27.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:27 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:27.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:27 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:27.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:27 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:27.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:27 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:27.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:27 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:27.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:27 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:27.714 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:27 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[61192]: 2024-03-24T11:22:27.383+0000 7fbb30ba5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-24T11:22:27.714 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:27 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[61192]: [24/Mar/2024:11:22:27] ENGINE Bus STARTING 2024-03-24T11:22:27.714 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:27 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[61192]: CherryPy Checker: 2024-03-24T11:22:27.714 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:27 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[61192]: The Application mounted at '' has an empty config. 2024-03-24T11:22:27.714 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:27 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[61192]: 2024-03-24T11:22:27.714 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:27 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[61192]: [24/Mar/2024:11:22:27] ENGINE Serving on http://:::9283 2024-03-24T11:22:27.715 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:27 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[61192]: [24/Mar/2024:11:22:27] ENGINE Bus STARTED 2024-03-24T11:22:28.422 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-03-24T11:22:28.422 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-03-24T11:22:28.423 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: pgmap v19: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 471 B/s rd, 0 op/s 2024-03-24T11:22:28.423 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:28.423 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:28.423 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:28.423 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:28.423 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:28.423 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:28.423 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:28.423 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:28.423 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:28.423 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:28.423 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:28.423 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:28.424 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:28.424 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: Standby manager daemon x restarted 2024-03-24T11:22:28.424 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: Standby manager daemon x started 2024-03-24T11:22:28.424 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.? 172.21.15.139:0/939167917' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-24T11:22:28.424 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.? 172.21.15.139:0/939167917' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T11:22:28.424 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/959821378' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-24T11:22:28.424 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.? 172.21.15.139:0/939167917' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-24T11:22:28.424 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.? 172.21.15.139:0/939167917' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T11:22:28.424 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:28.424 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:28.424 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:28.424 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:28.425 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:28.425 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-03-24T11:22:28.425 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-03-24T11:22:28.425 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: pgmap v19: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 471 B/s rd, 0 op/s 2024-03-24T11:22:28.425 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:28.425 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:28.425 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:28.425 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:28.425 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:28.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:28.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:28.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:28.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:28.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:28.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:28.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:28.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:28.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: Standby manager daemon x restarted 2024-03-24T11:22:28.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: Standby manager daemon x started 2024-03-24T11:22:28.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.? 172.21.15.139:0/939167917' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-24T11:22:28.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.? 172.21.15.139:0/939167917' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T11:22:28.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/959821378' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-24T11:22:28.427 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.? 172.21.15.139:0/939167917' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-24T11:22:28.427 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.? 172.21.15.139:0/939167917' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T11:22:28.427 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:28.427 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:28.427 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:28.427 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:28.427 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:28.427 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:28.427 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:28.427 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:28.428 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:28.428 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:28.428 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:28.428 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:28.428 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:28.428 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:28.428 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:28.428 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:28.428 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:28.428 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:28.428 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:28.428 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:28.428 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:28.428 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:28.428 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:28.429 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:28.429 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:28.429 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-03-24T11:22:28.429 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: Cluster is now healthy 2024-03-24T11:22:28.429 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:28.429 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:28.429 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:28.429 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:28.429 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:28.429 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:28.429 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:28.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:28.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:28.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:28.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:28.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:28.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:28.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:28.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:28.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:28.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:28.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:28.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:28.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:28.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:28.431 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:28.431 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:28.431 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:28.431 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:28.431 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:28.431 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:28.431 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-03-24T11:22:28.431 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: Cluster is now healthy 2024-03-24T11:22:28.431 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:28.432 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:28.432 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:28.432 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:28.432 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:28.432 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:28.432 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:28.432 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:28.432 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:28.432 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:28.433 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:28.433 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:28.433 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:28.433 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:28.433 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:28.433 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:28.433 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:28.433 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:28.433 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:28.433 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:28.433 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:28.433 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:28.434 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:28.434 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:28.434 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:28.434 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:28.434 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:28.434 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:28.434 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:28.434 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:28.434 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:28.434 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:28.434 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:28 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:28.484 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-03-24T11:22:28.484 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-03-24T11:22:28.484 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: pgmap v19: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 471 B/s rd, 0 op/s 2024-03-24T11:22:28.484 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:28.485 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:28.485 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:28.485 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:28.485 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:28.485 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:28.485 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:28.485 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:28.485 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:28.485 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:28.485 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:28.485 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:28.485 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:28.486 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: Standby manager daemon x restarted 2024-03-24T11:22:28.486 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: Standby manager daemon x started 2024-03-24T11:22:28.486 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.? 172.21.15.139:0/939167917' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-24T11:22:28.486 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.? 172.21.15.139:0/939167917' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T11:22:28.486 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/959821378' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-24T11:22:28.486 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.? 172.21.15.139:0/939167917' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-24T11:22:28.486 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.? 172.21.15.139:0/939167917' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T11:22:28.486 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:28.486 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:28.486 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:28.486 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:28.486 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:28.486 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:28.486 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:28.487 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:28.487 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:28.487 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:28.487 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:28.487 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:28.487 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:28.487 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:28.487 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:28.487 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:28.487 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:28.487 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:28.487 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:28.487 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:28.488 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:28.488 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:28.488 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:28.488 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:28.488 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:28.488 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-03-24T11:22:28.488 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: Cluster is now healthy 2024-03-24T11:22:28.489 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:28.489 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:28.489 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:28.489 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:28.489 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:28.489 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:28.489 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:28.489 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:28.489 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:28.489 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:28.489 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:28.489 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:28.489 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:28.489 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:28.490 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:28.490 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:28.490 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:28.490 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:28.490 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:28.490 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:28.857 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:28 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:22:28.659Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"smithi062:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"172.21.15.139:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-24T11:22:29.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:29.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:29.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:29.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:29.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:29.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:29.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:29.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:29.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:29.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:29.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:29.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:29.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:29.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:29.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:29.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:29.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:29.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:29.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:29.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:29.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: mgrmap e34: y(active, since 26s), standbys: x 2024-03-24T11:22:29.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:29.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:29.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:29.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:29.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:29.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:29.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:29.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:29.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:29.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:29.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:29.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:29.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:29.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:29.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:29.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:29.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:29.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:29.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:29.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:29.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:29.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:29.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:29.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:29.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:29.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:29.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:29.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:29.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:29.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:29.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:29.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:29.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:29.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:29.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:29.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:29.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:29.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:29.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:29.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:29.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:29.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:29.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:29.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:29.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:29.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:29.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:29.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:29.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:29.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:29.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:29.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:29.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:29.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:29.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:29.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:29.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:29.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:29.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:29.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: mgrmap e34: y(active, since 26s), standbys: x 2024-03-24T11:22:29.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:29.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:29.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:29.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:29.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:29.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:29.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:29.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:29.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:29.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:29.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:29.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:29.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:29.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:29.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:29.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:29.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:29.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:29.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:29.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:29.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:29.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:29.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:29.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:29.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:29.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:29.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:29.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:29.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:29.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:29.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: mgrmap e34: y(active, since 26s), standbys: x 2024-03-24T11:22:29.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:29.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:29.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:29.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:29.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:29.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:29.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:29.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:29.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:29.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:29.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:29.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:29.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:29.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:29.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:29.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:29.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:29.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:29.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:29.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:29.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:29.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:29.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:29.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:29.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:29.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:29.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:29.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:29.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:29.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:29.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:29.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:29.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:29.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:29.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:29.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:29.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:29.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:29.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:29.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:29.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:29.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:29.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:29.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:29.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:29.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:29.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:29.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:29.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:29.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:29.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:29.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:29.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:29.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:29.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:29.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:29.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:29.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:29.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:29.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:29.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:29.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:29.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:29.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:29.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:29.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:29.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:29.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:30.397 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:30.397 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:30.398 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:30.398 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:30.398 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:30.398 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:30.398 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:30.398 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: pgmap v20: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 601 B/s rd, 0 op/s 2024-03-24T11:22:30.398 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:30.398 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:30.399 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:30.399 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:30.399 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:30.399 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:30.399 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:30.399 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:30.399 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:30.400 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:30.400 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:30.400 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:30.400 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:30.400 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:30.400 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:30.400 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:30.400 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:30.400 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:30.401 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:30.401 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:30.401 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:30.401 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:30.401 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:30.401 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:30.401 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:30.401 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:30.401 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:30.401 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:30.401 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:30.401 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:30.401 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:30.402 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:30.402 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:30.402 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:30.402 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:30.402 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:30.402 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:30.402 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:30.402 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:30.402 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:30.402 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:30.402 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:30.402 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:30.403 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:30.403 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:30.403 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:30.403 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:30.403 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:30.403 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:30 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:30.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:30.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:30.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:30.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:30.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:30.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:30.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:30.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: pgmap v20: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 601 B/s rd, 0 op/s 2024-03-24T11:22:30.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:30.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:30.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:30.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:30.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:30.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:30.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:30.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:30.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:30.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:30.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:30.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:30.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:30.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:30.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:30.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:30.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:30.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:30.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:30.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:30.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:30.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:30.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:30.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:30.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:30.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:30.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:30.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:30.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:30.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: pgmap v20: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 601 B/s rd, 0 op/s 2024-03-24T11:22:30.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:30.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:30.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:30.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:30.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:30.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:30.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:30.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:30.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:30.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:30.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:30.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:30.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:30.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:30.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:30.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:30.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:30.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:30.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:30.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:30.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:30.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:30.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:30.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:30.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:30.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:30.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:30.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:30.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:30.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:30.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:30.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:30.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:30.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:30.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:30.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:30.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:30.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:30.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:30.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:30.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:30.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:30.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:30.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:30.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:30.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:30.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:30.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:30.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:30.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:30.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:30.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:30.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:30.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:30.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:30.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:30.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:30.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:30.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:30.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:30.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:30.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:30.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:30.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:30.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:30.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:30.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:30.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:30.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:30.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:30.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:30.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:30.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:30.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:30.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:30.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:31.127 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:30 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:22:30] "GET /metrics HTTP/1.1" 200 38315 "" "Prometheus/2.33.4" 2024-03-24T11:22:31.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:31 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:31.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:31 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:31.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:31 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:31.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:31 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:31.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:31 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:31.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:31 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:31.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:31 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:31.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:31 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:31.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:31 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:31.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:31 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:31.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:31 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:31.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:31 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:31.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:31 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:31.424 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:31 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:31.425 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:31 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:31.425 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:31 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:31.425 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:31 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:31.425 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:31 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:31.425 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:31 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:31.425 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:31 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:31.425 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:31 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:31.425 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:31 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:31.425 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:31 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:31.425 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:31 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:31.425 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:31 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:31.425 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:31 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:31.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:31 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:31.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:31 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:31.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:31 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:31.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:31 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:31.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:31 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:31.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:31 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:31.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:31 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:31.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:31 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:31.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:31 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:31.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:31 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:31.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:31 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:31.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:31 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:31.427 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:31 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:32.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:32 smithi062 ceph-mon[29666]: pgmap v21: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 601 B/s rd, 0 op/s 2024-03-24T11:22:32.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:32 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:32.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:32 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:22:32.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:32 smithi062 ceph-mon[24799]: pgmap v21: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 601 B/s rd, 0 op/s 2024-03-24T11:22:32.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:32 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:32.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:32 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:22:32.591 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:32 smithi139 ceph-mon[25719]: pgmap v21: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 601 B/s rd, 0 op/s 2024-03-24T11:22:32.592 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:32 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:32.592 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:32 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:22:34.278 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:34 smithi139 ceph-mon[25719]: pgmap v22: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 233 B/s rd, 0 op/s 2024-03-24T11:22:34.278 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:34 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T11:22:34.278 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:34 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T11:22:34.278 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:34 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:34.476 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:34 smithi062 ceph-mon[29666]: pgmap v22: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 233 B/s rd, 0 op/s 2024-03-24T11:22:34.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:34 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T11:22:34.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:34 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T11:22:34.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:34 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:34.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:34 smithi062 ceph-mon[24799]: pgmap v22: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 233 B/s rd, 0 op/s 2024-03-24T11:22:34.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:34 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T11:22:34.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:34 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T11:22:34.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:34 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:35.327 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:35 smithi139 ceph-mon[25719]: Upgrade: Updating mgr.x 2024-03-24T11:22:35.327 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:35 smithi139 ceph-mon[25719]: Deploying daemon mgr.x on smithi139 2024-03-24T11:22:35.327 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:35 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:35.327 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:35 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:35.327 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:35 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:35.476 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:35 smithi062 ceph-mon[29666]: Upgrade: Updating mgr.x 2024-03-24T11:22:35.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:35 smithi062 ceph-mon[29666]: Deploying daemon mgr.x on smithi139 2024-03-24T11:22:35.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:35 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:35.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:35 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:35.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:35 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:35.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:35 smithi062 ceph-mon[24799]: Upgrade: Updating mgr.x 2024-03-24T11:22:35.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:35 smithi062 ceph-mon[24799]: Deploying daemon mgr.x on smithi139 2024-03-24T11:22:35.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:35 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:35.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:35 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:35.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:35 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:35.562 INFO:teuthology.orchestra.run.smithi062.stdout:true 2024-03-24T11:22:36.223 INFO:teuthology.orchestra.run.smithi062.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T11:22:36.224 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi062 smithi062 running 1s ago 12m - - 2024-03-24T11:22:36.224 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi139 smithi139 running 6s ago 11m - - 2024-03-24T11:22:36.224 INFO:teuthology.orchestra.run.smithi062.stdout:alertmanager.a smithi062 *:9093,9094 running (112s) 1s ago 6m 17.7M - 0.23.0 ba2b418f427c 818298d58140 2024-03-24T11:22:36.224 INFO:teuthology.orchestra.run.smithi062.stdout:grafana.a smithi139 *:3000 running (70s) 6s ago 6m 45.5M - 8.3.5 dad864ee21e9 b48167e7171e 2024-03-24T11:22:36.224 INFO:teuthology.orchestra.run.smithi062.stdout:iscsi.foo.smithi062.dggsvk smithi062 running (9s) 1s ago 5m 50.0M - 3.5 e1d6a67b021e 0a98806ec839 2024-03-24T11:22:36.224 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.x smithi139 *:8443,9283 running (2m) 6s ago 11m 353M - 19.0.0-1667-gdb0330b1 4f325f8440e9 ed2c6bf552c0 2024-03-24T11:22:36.224 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.y smithi062 *:8443,9283,8765 running (42s) 1s ago 12m 502M - 19.0.0-1667-gdb0330b1 4f325f8440e9 ca1a49028d65 2024-03-24T11:22:36.224 INFO:teuthology.orchestra.run.smithi062.stdout:mon.a smithi062 running (12m) 1s ago 13m 62.5M 2048M 17.2.0 e1d6a67b021e 712f01aa9180 2024-03-24T11:22:36.224 INFO:teuthology.orchestra.run.smithi062.stdout:mon.b smithi139 running (11m) 6s ago 11m 56.4M 2048M 17.2.0 e1d6a67b021e 8945cdb4e6aa 2024-03-24T11:22:36.224 INFO:teuthology.orchestra.run.smithi062.stdout:mon.c smithi062 running (11m) 1s ago 11m 52.4M 2048M 17.2.0 e1d6a67b021e 39110a16ab62 2024-03-24T11:22:36.224 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.a smithi062 *:9100 running (2m) 1s ago 7m 18.2M - 1.3.1 1dbe0e931976 79d5a02a999f 2024-03-24T11:22:36.224 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.b smithi139 *:9100 running (85s) 6s ago 7m 8242k - 1.3.1 1dbe0e931976 8f9f4608e465 2024-03-24T11:22:36.224 INFO:teuthology.orchestra.run.smithi062.stdout:osd.0 smithi062 running (10m) 1s ago 10m 52.8M 4096M 17.2.0 e1d6a67b021e 8040b30d7170 2024-03-24T11:22:36.224 INFO:teuthology.orchestra.run.smithi062.stdout:osd.1 smithi062 running (10m) 1s ago 10m 54.7M 4096M 17.2.0 e1d6a67b021e e62d22e5dc29 2024-03-24T11:22:36.225 INFO:teuthology.orchestra.run.smithi062.stdout:osd.2 smithi062 running (10m) 1s ago 10m 48.1M 4096M 17.2.0 e1d6a67b021e 714f550103a7 2024-03-24T11:22:36.225 INFO:teuthology.orchestra.run.smithi062.stdout:osd.3 smithi062 running (9m) 1s ago 9m 52.5M 4096M 17.2.0 e1d6a67b021e 1d2868bd5195 2024-03-24T11:22:36.225 INFO:teuthology.orchestra.run.smithi062.stdout:osd.4 smithi139 running (9m) 6s ago 9m 56.5M 4096M 17.2.0 e1d6a67b021e 7568bfa9e74d 2024-03-24T11:22:36.225 INFO:teuthology.orchestra.run.smithi062.stdout:osd.5 smithi139 running (9m) 6s ago 9m 50.1M 4096M 17.2.0 e1d6a67b021e dd0abd599a14 2024-03-24T11:22:36.225 INFO:teuthology.orchestra.run.smithi062.stdout:osd.6 smithi139 running (8m) 6s ago 8m 50.2M 4096M 17.2.0 e1d6a67b021e a4b1289aca69 2024-03-24T11:22:36.225 INFO:teuthology.orchestra.run.smithi062.stdout:osd.7 smithi139 running (8m) 6s ago 8m 52.1M 4096M 17.2.0 e1d6a67b021e bc7fadbc1384 2024-03-24T11:22:36.225 INFO:teuthology.orchestra.run.smithi062.stdout:prometheus.a smithi139 *:9095 running (90s) 6s ago 7m 58.8M - 2.33.4 514e6a882f6e 5c0c71f4d276 2024-03-24T11:22:36.225 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi062.sacpds smithi062 *:8000 running (5m) 1s ago 5m 72.3M - 17.2.0 e1d6a67b021e 89f4ac308e98 2024-03-24T11:22:36.225 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi139.yljesh smithi139 *:8000 running (5m) 6s ago 5m 70.7M - 17.2.0 e1d6a67b021e e189bdc12cae 2024-03-24T11:22:36.225 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi062.xvpxnk smithi062 *:80 running (5m) 1s ago 5m 72.0M - 17.2.0 e1d6a67b021e d1aa74adab44 2024-03-24T11:22:36.225 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi139.pbtrmp smithi139 *:80 running (5m) 6s ago 5m 74.4M - 17.2.0 e1d6a67b021e 7f8e336755d7 2024-03-24T11:22:36.269 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:36 smithi139 ceph-mon[25719]: pgmap v23: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 665 B/s rd, 0 op/s 2024-03-24T11:22:36.476 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:36 smithi062 ceph-mon[29666]: pgmap v23: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 665 B/s rd, 0 op/s 2024-03-24T11:22:36.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:36 smithi062 ceph-mon[24799]: pgmap v23: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 665 B/s rd, 0 op/s 2024-03-24T11:22:36.628 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:22:36.628 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": { 2024-03-24T11:22:36.628 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 3 2024-03-24T11:22:36.628 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:22:36.628 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": { 2024-03-24T11:22:36.629 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-24T11:22:36.629 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:22:36.629 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": { 2024-03-24T11:22:36.629 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 8 2024-03-24T11:22:36.629 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:22:36.629 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": {}, 2024-03-24T11:22:36.629 INFO:teuthology.orchestra.run.smithi062.stdout: "rgw": { 2024-03-24T11:22:36.629 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4 2024-03-24T11:22:36.629 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:22:36.629 INFO:teuthology.orchestra.run.smithi062.stdout: "overall": { 2024-03-24T11:22:36.629 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 15, 2024-03-24T11:22:36.629 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-24T11:22:36.629 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-24T11:22:36.629 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:22:36.964 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:22:36.964 INFO:teuthology.orchestra.run.smithi062.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69", 2024-03-24T11:22:36.964 INFO:teuthology.orchestra.run.smithi062.stdout: "in_progress": true, 2024-03-24T11:22:36.964 INFO:teuthology.orchestra.run.smithi062.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T11:22:36.964 INFO:teuthology.orchestra.run.smithi062.stdout: "services_complete": [ 2024-03-24T11:22:36.964 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr" 2024-03-24T11:22:36.965 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-03-24T11:22:36.965 INFO:teuthology.orchestra.run.smithi062.stdout: "progress": "2/23 daemons upgraded", 2024-03-24T11:22:36.965 INFO:teuthology.orchestra.run.smithi062.stdout: "message": "Currently upgrading mgr daemons", 2024-03-24T11:22:36.965 INFO:teuthology.orchestra.run.smithi062.stdout: "is_paused": false 2024-03-24T11:22:36.965 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:22:37.256 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:37 smithi139 ceph-mon[25719]: from='client.25402 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:22:37.256 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:37 smithi139 ceph-mon[25719]: from='client.25408 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:22:37.256 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:37 smithi139 ceph-mon[25719]: from='client.15510 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:22:37.256 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:37 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/3589520414' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:37.256 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:37 smithi139 ceph-mon[25719]: pgmap v24: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 665 B/s rd, 0 op/s 2024-03-24T11:22:37.256 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:37 smithi139 ceph-mon[25719]: from='client.15522 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:22:37.256 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:37 smithi139 ceph-mon[25719]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:22:37.368 INFO:teuthology.orchestra.run.smithi062.stdout:HEALTH_OK 2024-03-24T11:22:37.608 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:37 smithi139 systemd[1]: Stopping Ceph mgr.x for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:22:37.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:37 smithi062 ceph-mon[24799]: from='client.25402 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:22:37.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:37 smithi062 ceph-mon[24799]: from='client.25408 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:22:37.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:37 smithi062 ceph-mon[24799]: from='client.15510 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:22:37.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:37 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/3589520414' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:37.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:37 smithi062 ceph-mon[24799]: pgmap v24: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 665 B/s rd, 0 op/s 2024-03-24T11:22:37.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:37 smithi062 ceph-mon[24799]: from='client.15522 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:22:37.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:37 smithi062 ceph-mon[24799]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:22:37.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:37 smithi062 ceph-mon[29666]: from='client.25402 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:22:37.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:37 smithi062 ceph-mon[29666]: from='client.25408 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:22:37.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:37 smithi062 ceph-mon[29666]: from='client.15510 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:22:37.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:37 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/3589520414' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:37.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:37 smithi062 ceph-mon[29666]: pgmap v24: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 665 B/s rd, 0 op/s 2024-03-24T11:22:37.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:37 smithi062 ceph-mon[29666]: from='client.15522 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:22:37.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:37 smithi062 ceph-mon[29666]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:22:38.529 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:38 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/3287821153' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:22:38.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:38 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/3287821153' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:22:38.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:38 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/3287821153' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:22:38.837 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:38 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:22:38.658Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"smithi062:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"172.21.15.139:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-24T11:22:38.837 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:38 smithi139 podman[63599]: 2024-03-24 11:22:38.52922035 +0000 UTC m=+0.816954332 container died 416473ec6776de2084e44695044ffc9d600d44019d24ef77e5725b3592bd5c02 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T11:22:39.108 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:38 smithi139 podman[63599]: 2024-03-24 11:22:38.837318327 +0000 UTC m=+1.125052306 container cleanup 416473ec6776de2084e44695044ffc9d600d44019d24ef77e5725b3592bd5c02 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x, RELEASE=squid-db0330b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-24T11:22:39.108 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:38 smithi139 bash[63599]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x 2024-03-24T11:22:39.108 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:39 smithi139 podman[63656]: 2024-03-24 11:22:39.047087095 +0000 UTC m=+0.848832943 container remove 416473ec6776de2084e44695044ffc9d600d44019d24ef77e5725b3592bd5c02 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-24T11:22:39.108 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:39 smithi139 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mgr.x.service: Main process exited, code=exited, status=143/n/a 2024-03-24T11:22:39.411 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:39 smithi139 ceph-mon[25719]: pgmap v25: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-03-24T11:22:39.411 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:39 smithi139 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mgr.x.service: Failed with result 'exit-code'. 2024-03-24T11:22:39.411 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:39 smithi139 systemd[1]: Stopped Ceph mgr.x for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:22:39.411 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:39 smithi139 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mgr.x.service: Consumed 8.695s CPU time. 2024-03-24T11:22:39.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:39 smithi062 ceph-mon[24799]: pgmap v25: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-03-24T11:22:39.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:39 smithi062 ceph-mon[29666]: pgmap v25: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-03-24T11:22:39.740 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:39 smithi139 systemd[1]: Starting Ceph mgr.x for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:22:40.000 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:39 smithi139 podman[63959]: 2024-03-24 11:22:39.646411337 +0000 UTC m=+0.023364039 image pull e1d6a67b021eb077ee22bf650f1a9fb1980a2cf5c36bdb9cba9eac6de8f702d9 quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a 2024-03-24T11:22:40.000 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:39 smithi139 podman[63959]: 2024-03-24 11:22:39.756356991 +0000 UTC m=+0.133309691 container create 9079b49d78414e4cd7622dc086e392dd568de76ebe04ae61ef70378aa2f41839 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x, GIT_REPO=https://github.com/ceph/ceph-container.git, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, io.buildah.version=1.19.8, com.redhat.component=centos-stream-container, io.openshift.tags=base centos centos-stream, release=754, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, build-date=2022-05-03T08:36:31.336870, vcs-type=git, version=8, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, name=centos-stream, GIT_BRANCH=HEAD, distribution-scope=public, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, architecture=x86_64, io.openshift.expose-services=, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., RELEASE=HEAD, vendor=Red Hat, Inc., io.k8s.display-name=CentOS Stream 8, CEPH_POINT_RELEASE=-17.2.0) 2024-03-24T11:22:40.000 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:39 smithi139 podman[63959]: 2024-03-24 11:22:39.976009092 +0000 UTC m=+0.352961809 container init 9079b49d78414e4cd7622dc086e392dd568de76ebe04ae61ef70378aa2f41839 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x, GIT_CLEAN=True, architecture=x86_64, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., release=754, CEPH_POINT_RELEASE=-17.2.0, com.redhat.component=centos-stream-container, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, io.buildah.version=1.19.8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, RELEASE=HEAD, vendor=Red Hat, Inc., io.openshift.expose-services=, build-date=2022-05-03T08:36:31.336870, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_BRANCH=HEAD, vcs-type=git, version=8, io.openshift.tags=base centos centos-stream, name=centos-stream, io.k8s.display-name=CentOS Stream 8, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, distribution-scope=public, ceph=True, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac) 2024-03-24T11:22:40.000 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:39 smithi139 podman[63959]: 2024-03-24 11:22:39.978872169 +0000 UTC m=+0.355824871 container start 9079b49d78414e4cd7622dc086e392dd568de76ebe04ae61ef70378aa2f41839 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, io.openshift.tags=base centos centos-stream, name=centos-stream, io.k8s.display-name=CentOS Stream 8, architecture=x86_64, RELEASE=HEAD, io.buildah.version=1.19.8, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, distribution-scope=public, com.redhat.component=centos-stream-container, build-date=2022-05-03T08:36:31.336870, vcs-type=git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-17.2.0, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, version=8, io.openshift.expose-services=, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., com.redhat.license_terms=https://centos.org/legal/licensing-policy/, vendor=Red Hat, Inc., release=754, GIT_REPO=https://github.com/ceph/ceph-container.git) 2024-03-24T11:22:40.358 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:40 smithi139 ceph-mgr[64004]: -- 172.21.15.139:0/3585549382 <== mon.2 v2:172.21.15.139:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555faa80e340 con 0x555faa7e7000 2024-03-24T11:22:40.358 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:40 smithi139 bash[63959]: 9079b49d78414e4cd7622dc086e392dd568de76ebe04ae61ef70378aa2f41839 2024-03-24T11:22:40.358 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:40 smithi139 systemd[1]: Started Ceph mgr.x for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:22:40.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:40 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:40.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:40 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:40.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:40 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:40.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:40 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:40.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:40 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:40.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:40 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:40.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:40 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:40.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:40 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:40.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:40 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:40.726 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:40 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:40.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:40 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:40.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:40 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:40.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:40 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:40.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:40 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:40.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:40 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:40.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:40 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:40.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:40 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:40.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:40 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:40.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:40 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:40.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:40 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:40.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:40 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:40.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:40 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:40.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:40 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:40.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:40 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:40.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:40 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:40.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:40 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:40.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:40 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:40.857 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:40 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[63987]: 2024-03-24T11:22:40.524+0000 7f2efe927000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-24T11:22:40.858 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:40 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[63987]: 2024-03-24T11:22:40.621+0000 7f2efe927000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-24T11:22:41.226 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:40 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:22:40] "GET /metrics HTTP/1.1" 200 38316 "" "Prometheus/2.33.4" 2024-03-24T11:22:41.358 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[63987]: 2024-03-24T11:22:41.274+0000 7f2efe927000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-24T11:22:41.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:41.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:41.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:41.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:41.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:41.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:41.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:41.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:41.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:41.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:41.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:41.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:41.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:41.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:41.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:41.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:41.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:41.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:41.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:41.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:41.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:41.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:41.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:41.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:41.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:41.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:41.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:41.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:41.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:41.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:41.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:41.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:41.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:41.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:41.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:41.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:41.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:41.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:41.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:41.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:41.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:41.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:41.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: pgmap v26: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:22:41.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:41.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:41.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:41.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:41.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:41.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:41.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:41.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:41.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:41.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:41.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:41.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:41.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:41.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:41.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:41.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:41.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:41.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:41.696 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[63987]: 2024-03-24T11:22:41.366+0000 7f2efe927000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-24T11:22:41.697 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[63987]: 2024-03-24T11:22:41.455+0000 7f2efe927000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-24T11:22:41.697 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[63987]: 2024-03-24T11:22:41.610+0000 7f2efe927000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-24T11:22:41.697 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[63987]: 2024-03-24T11:22:41.696+0000 7f2efe927000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-24T11:22:41.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:41.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:41.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:41.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:41.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:41.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:41.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:41.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:41.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:41.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:41.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:41.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:41.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:41.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:41.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:41.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:41.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:41.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:41.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:41.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:41.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:41.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:41.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:41.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:41.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:41.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:41.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:41.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:41.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:41.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:41.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:41.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:41.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:41.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:41.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:41.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:41.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:41.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:41.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:41.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:41.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:41.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:41.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:41.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:41.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:41.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:41.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:41.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:41.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:41.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:41.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:41.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:41.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:41.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:41.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:41.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:41.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:41.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:41.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:41.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:41.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:41.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:41.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:41.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:41.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:41.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:41.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:41.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:41.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:41.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:41.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:41.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: pgmap v26: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:22:41.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:41.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:41.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:41.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:41.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:41.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:41.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:41.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:41.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:41.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:41.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:41.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:41.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:41.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:41.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:41.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:41.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:41.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:41.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:41.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:41.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:41.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:41.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:41.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:41.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:41.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:41.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:41.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:41.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:41.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:41.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:41.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: pgmap v26: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:22:41.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:41.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:41.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:41.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:41.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:41.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:41.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:41.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:41.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:41.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:41.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:41.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:41.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:41.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:41.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:41.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:41.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:41.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:41 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:42.081 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[63987]: 2024-03-24T11:22:41.822+0000 7f2efe927000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-24T11:22:42.081 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:41 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[63987]: 2024-03-24T11:22:41.910+0000 7f2efe927000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-24T11:22:42.081 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[63987]: 2024-03-24T11:22:42.080+0000 7f2efe927000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-24T11:22:42.358 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[63987]: 2024-03-24T11:22:42.171+0000 7f2efe927000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-24T11:22:42.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:42.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:42.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:42.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:42.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:42.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:42.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:42.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:42.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:42.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:42.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:42.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:42.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:42.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:42.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:42.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:42.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:42.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:42.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:42.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:42.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:42.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:42.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:42.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:42.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:42.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:42.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:42.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:42.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:42.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:42.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:42.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:42.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:42.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:42.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:42.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:42.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:42.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:42.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:42.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:42.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:42.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:42.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:42.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:42.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:42.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:42.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:42.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:42.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:42.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:42.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:42.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:42.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:42.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:42.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:42.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:42.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:42.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:42.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:42.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:42.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:42.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:42.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:42.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:42.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:42.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:42 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:42.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:42.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:42.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:42.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:42.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:42.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:42.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:42.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:42.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:42.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:42.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:42.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:42.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:42.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:42.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:42.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:42.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:42.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:42.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:42.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:42.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:42.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:42.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:42.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:42.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:42.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:42.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:42.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:42.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:42.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:42.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:42.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:42.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:42.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:42.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:42.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:42.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:42.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:42.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:42.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:42.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:42.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:42.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:42.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:42.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:42.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:42.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:42.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:42.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:42.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:42.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:42.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:42.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:42.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:42.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:42.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:42.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:42.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:42.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:42.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:42.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:42.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:42.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:42.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:42.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:42.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:42.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:42.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:42.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:42.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:42.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:42.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:42.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:42.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:42.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:42.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:42.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:42.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:42.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:42.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:42.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:42.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:42.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:42.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:42.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:42.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:42.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:42.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:42.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:42.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:42.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:42.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:42.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:42.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:42.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:42.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:42.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:42.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:42.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:42.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:42.739 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:42.739 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:42.739 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:42.739 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:42.739 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:42.739 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:42.739 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:42.739 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:42.739 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:42.739 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:42.739 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:42.739 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:42.739 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:42.739 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:42.739 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:42.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:42.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:42.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:42.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:42.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:42.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:42.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:42.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:42.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:42.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:42.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:42.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:42.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:42.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:42.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:42.741 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:42.741 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:42 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:43.460 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[63987]: 2024-03-24T11:22:43.318+0000 7f2efe927000 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-24T11:22:43.460 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[63987]: 2024-03-24T11:22:43.459+0000 7f2efe927000 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-24T11:22:43.460 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:43.460 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:43.460 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:43.460 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:43.461 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:43.461 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:43.461 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:43.461 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:43.461 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:43.461 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:43.461 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:43.461 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:43.461 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:43.461 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:43.461 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:43.461 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:43.461 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:43.461 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:43.462 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:43.462 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:43.462 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:43.462 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:43.462 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:43.462 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:43.462 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:43.462 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:43.462 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:43.462 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:43.462 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:43.462 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:43.462 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:43.463 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:43.463 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:43.463 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:43.463 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:43.463 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:43.463 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:43.463 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:43.463 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:43.463 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:43.463 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:43.463 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:43.463 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:43.464 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:43.464 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:43.464 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:43.464 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:43.464 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:43.464 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: pgmap v27: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:22:43.464 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:43.464 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:43.464 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:43.464 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:43.464 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:43.464 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:43.464 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:43.464 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:43.464 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:43.465 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:43.465 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:43.465 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:43.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:43.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:43.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:43.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:43.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:43.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:43.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:43.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:43.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:43.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:43.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:43.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:43.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:43.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:43.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:43.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:43.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:43.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:43.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:43.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:43.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:43.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:43.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:43.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:43.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:43.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:43.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:43.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:43.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:43.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:43.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:43.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:43.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:43.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:43.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:43.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:43.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:43.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:43.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:43.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:43.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:43.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:43.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:43.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:43.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:43.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:43.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:43.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:43.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:43.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:43.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:43.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:43.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:43.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:43.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:43.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:43.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:43.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:43.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:43.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:43.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:43.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:43.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:43.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:43.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:43.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:43.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:43.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:43.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:43.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:43.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:43.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:43.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:43.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:43.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:43.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:43.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:43.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: pgmap v27: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:22:43.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:43.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:43.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:43.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:43.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:43.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:43.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:43.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:43.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:43.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:43.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:43.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:43.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:43.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:43.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:43.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:43.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:43.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:43.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:43.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:43.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:43.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:43.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:43.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:43.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:43.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:43.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:43.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:43.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:43.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:43.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:43.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: pgmap v27: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:22:43.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:43.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:43.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:43.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:43.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:43.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:43.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:43.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:43.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:43.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:43.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:43.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:43 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:43.770 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[63987]: 2024-03-24T11:22:43.677+0000 7f2efe927000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-24T11:22:43.770 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[63987]: 2024-03-24T11:22:43.769+0000 7f2efe927000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-24T11:22:44.059 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:43 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[63987]: 2024-03-24T11:22:43.929+0000 7f2efe927000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-24T11:22:44.319 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[63987]: 2024-03-24T11:22:44.269+0000 7f2efe927000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-24T11:22:44.320 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:44.320 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:44.320 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:44.320 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:44.320 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:44.320 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:44.320 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:44.320 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:44.320 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:44.320 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:44.320 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:44.320 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:44.320 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:44.321 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:44.321 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:44.321 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:44.321 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:44.321 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:44.321 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:44.321 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:44.321 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:44.321 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:44.321 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:44.321 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:44.321 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:44.321 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:44.321 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:44.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:44.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:44.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:44.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:44.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:44.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:44.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:44.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:44.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:44.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:44.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:44.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:44.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:44.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:44.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:44.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:44.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:44.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:44.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:44.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:44.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:44.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:44.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:44.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:44.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:44.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:44.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:44.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:44.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:44.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:44.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:44.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:44.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:44.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:44.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:44.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:44.325 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:44.325 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:44.325 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:44.325 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:44.325 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:44.325 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:44.325 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:44.325 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:44.325 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:44.607 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[63987]: 2024-03-24T11:22:44.501+0000 7f2efe927000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-24T11:22:44.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:44.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:44.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:44.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:44.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:44.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:44.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:44.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:44.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:44.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:44.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:44.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:44.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:44.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:44.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:44.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:44.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:44.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:44.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:44.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:44.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:44.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:44.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:44.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:44.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:44.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:44.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:44.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:44.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:44.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:44.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:44.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:44.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:44.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:44.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:44.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:44.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:44.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:44.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:44.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:44.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:44.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:44.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:44.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:44.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:44.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:44.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:44.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:44.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:44.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:44.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:44.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:44.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:44.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:44.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:44.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:44.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:44.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:44.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:44.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:44.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:44.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:44.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:44.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:44.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:44.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:44.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:44.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:44.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:44.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:44.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:44.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:44.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:44.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:44.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:44.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:44.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:44.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:44.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:44.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:44.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:44.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:44.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:44.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:44.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:44.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:44.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:44.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:44.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:44.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:44.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:44.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:44.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:44.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:44.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:44.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:44.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:44.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:44.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:44.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:44.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:44.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:44.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:44.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:44.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:44.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:44.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:44.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:44.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:44.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:44.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:44.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:44.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:44.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:44.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:44.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:44.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:44.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:44.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:44.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:44.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:44.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:44.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:44.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:44.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:44.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:44.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:44.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:44.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:44.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:44.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:44.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:44.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:44.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:44.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:44.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:44.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:44.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:44.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:44.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:44.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:44.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:44.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:44.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:44 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:44.857 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[63987]: 2024-03-24T11:22:44.607+0000 7f2efe927000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-24T11:22:44.858 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:44 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[63987]: 2024-03-24T11:22:44.786+0000 7f2efe927000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-24T11:22:45.294 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:45.294 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:45.294 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:45.294 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:45.294 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:45.294 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:45.294 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:45.294 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:45.294 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:45.295 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:45.295 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:45.295 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:45.295 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:45.295 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:45.295 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:45.295 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:45.295 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:45.295 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:45.295 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:45.295 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:45.295 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:45.295 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:45.295 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:45.295 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:45.295 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:45.296 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:45.296 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:45.296 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:45.296 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:45.296 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:45.296 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:45.296 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:45.296 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:45.296 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:45.297 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:45.297 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:45.297 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:45.297 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:45.297 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:45.297 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:45.297 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:45.297 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:45.297 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: pgmap v28: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:22:45.297 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:45.297 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:45.297 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:45.297 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:45.297 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:45.297 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:45.297 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:45.297 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:45.298 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:45.298 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:45.298 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:45.298 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:45.298 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:45.298 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:45.298 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:45.298 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:45.298 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:45.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:45.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:45.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:45.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:45.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:45.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:45.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:45.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:45.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:45.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:45.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:45.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:45.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:45.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:45.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:45.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:45.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:45.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:45.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:45.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:45.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:45.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:45.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:45.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:45.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:45.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:45.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:45.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:45.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:45.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:45.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:45.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:45.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:45.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:45.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:45.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:45.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:45.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:45.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:45.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:45.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:45.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:45.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:45.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:45.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:45.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:45.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:45.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:45.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:45.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:45.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:45.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:45.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:45.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:45.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:45.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:45.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:45.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:45.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:45.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:45.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:45.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:45.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:45.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:45.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:45.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:45.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:45.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:45.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:45.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:45.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:45.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: pgmap v28: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:22:45.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:45.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:45.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:45.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:45.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:45.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:45.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:45.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:45.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:45.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:45.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:45.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:45.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:45.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:45.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:45.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:45.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:45.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:45.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:45.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:45.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:45.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:45.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:45.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:45.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:45.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:45.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:45.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:45.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:45.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:45.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: pgmap v28: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:22:45.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:45.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:45.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:45.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:45.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:45.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:45.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:45.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:45.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:45.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:45.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:45.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:45.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:45.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:45.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:45.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:45.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:45 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:45.857 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:45 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[63987]: 2024-03-24T11:22:45.766+0000 7f2efe927000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-24T11:22:46.338 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[63987]: 2024-03-24T11:22:46.077+0000 7f2efe927000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-24T11:22:46.338 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[63987]: 2024-03-24T11:22:46.180+0000 7f2efe927000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-24T11:22:46.338 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[63987]: 2024-03-24T11:22:46.337+0000 7f2efe927000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-24T11:22:46.339 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:46.339 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:46.339 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:46.339 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:46.339 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:46.339 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:46.339 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:46.339 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:46.339 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:46.339 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:46.339 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:46.339 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:46.340 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:46.340 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:46.340 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:46.340 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:46.340 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:46.340 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:46.340 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:46.340 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:46.340 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:46.340 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:46.340 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:46.340 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:46.340 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:46.340 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:46.341 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:46.341 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:46.341 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:46.341 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:46.341 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:46.341 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:46.341 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:46.341 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:46.341 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:46.341 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:46.341 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:46.341 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:46.341 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:46.341 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:46.342 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:46.342 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:46.342 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:46.608 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[63987]: [24/Mar/2024:11:22:46] ENGINE Bus STARTING 2024-03-24T11:22:46.608 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[63987]: CherryPy Checker: 2024-03-24T11:22:46.608 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[63987]: The Application mounted at '' has an empty config. 2024-03-24T11:22:46.608 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[63987]: 2024-03-24T11:22:46.608 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[63987]: [24/Mar/2024:11:22:46] ENGINE Serving on http://:::9283 2024-03-24T11:22:46.608 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:46 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[63987]: [24/Mar/2024:11:22:46] ENGINE Bus STARTED 2024-03-24T11:22:46.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:46.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:46.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:46.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:46.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:46.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:46.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:46.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:46.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:46.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:46.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:46.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:46.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:46.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:46.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:46.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:46.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:46.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:46.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:46.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:46.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:46.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:46.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:46.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:46.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:46.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:46.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:46.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:46.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:46.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:46.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:46.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:46.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:46.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:46.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:46.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:46.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:46.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:46.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:46.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:46.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:46.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:46.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:46.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:46.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:46.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:46.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:46.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:46.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:46.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:46.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:46.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:46.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:46.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:46.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:46.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:46.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:46.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:46.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:46.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:46.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:46.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:46.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:46.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:46.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:46.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:46.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:46.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:46.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:46.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:46.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:46.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:46.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:46.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:46.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:46.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:46.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:46.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:46.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:46.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:46.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:46.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:46.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:46.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:46.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:46.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:46 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:47.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:47.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:47.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:47.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:47.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:47.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:47.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: Standby manager daemon x restarted 2024-03-24T11:22:47.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: Standby manager daemon x started 2024-03-24T11:22:47.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: from='mgr.? 172.21.15.139:0/2' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-24T11:22:47.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: from='mgr.? 172.21.15.139:0/2' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T11:22:47.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: from='mgr.? 172.21.15.139:0/2' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-24T11:22:47.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: from='mgr.? 172.21.15.139:0/2' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T11:22:47.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:47.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:47.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:47.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:47.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:47.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:47.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:47.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:47.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:47.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:47.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:47.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:47.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:47.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:47.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:47.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:47.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:47.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:47.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:47.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:47.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:47.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:47.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:47.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:47.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: pgmap v29: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:22:47.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:47.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:47.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:47.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:47.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:47.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:47.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:47.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:47.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:47.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:47.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:47.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:47.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:22:47.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:47.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:47.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:47.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:47.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:22:47.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:47.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:47.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:47.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:47.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:47.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:47.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: Standby manager daemon x restarted 2024-03-24T11:22:47.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: Standby manager daemon x started 2024-03-24T11:22:47.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: from='mgr.? 172.21.15.139:0/2' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-24T11:22:47.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: from='mgr.? 172.21.15.139:0/2' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T11:22:47.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: from='mgr.? 172.21.15.139:0/2' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-24T11:22:47.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: from='mgr.? 172.21.15.139:0/2' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T11:22:47.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:47.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:47.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:47.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:47.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:47.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:47.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:47.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:47.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:47.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:47.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:47.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:47.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:47.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:47.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:47.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:47.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:47.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:47.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:47.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:47.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:47.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:47.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:47.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:47.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: pgmap v29: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:22:47.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:47.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:47.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:47.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:47.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:47.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:47.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:47.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:47.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:47.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:47.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:47.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:47.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:22:47.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:47.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:47.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:47.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:47.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:47 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:22:47.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:47.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:47.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:47.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:47.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:47.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:47.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: Standby manager daemon x restarted 2024-03-24T11:22:47.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: Standby manager daemon x started 2024-03-24T11:22:47.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: from='mgr.? 172.21.15.139:0/2' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-24T11:22:47.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: from='mgr.? 172.21.15.139:0/2' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T11:22:47.574 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: from='mgr.? 172.21.15.139:0/2' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-24T11:22:47.575 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: from='mgr.? 172.21.15.139:0/2' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T11:22:47.575 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:47.575 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:47.575 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:47.575 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:47.575 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:47.575 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:47.575 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:47.575 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:47.575 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:47.575 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:47.575 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:47.575 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:47.575 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:47.576 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:47.576 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:47.576 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:47.576 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:47.576 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:47.576 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:47.576 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:47.576 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:47.576 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:47.577 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:47.577 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:47.577 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: pgmap v29: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:22:47.577 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:47.577 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:47.577 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:47.577 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:47.577 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:47.577 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:47.577 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:47.577 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:47.577 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:47.577 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:47.577 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:47.578 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:47.578 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:22:47.578 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:47.578 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:47.578 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:47.578 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:47.578 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:47 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:22:48.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:48 smithi139 ceph-mon[25719]: mgrmap e35: y(active, since 45s), standbys: x 2024-03-24T11:22:48.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:48 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:48.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:48 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:48.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:48 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:48.555 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:48 smithi062 ceph-mon[29666]: mgrmap e35: y(active, since 45s), standbys: x 2024-03-24T11:22:48.556 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:48 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:48.556 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:48 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:48.556 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:48 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:48.556 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:48 smithi062 ceph-mon[24799]: mgrmap e35: y(active, since 45s), standbys: x 2024-03-24T11:22:48.556 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:48 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:48.556 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:48 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:48.556 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:48 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:48.931 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:22:48.658Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"smithi062:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"172.21.15.139:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-24T11:22:49.408 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:49 smithi062 ceph-mon[29666]: pgmap v30: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:22:49.408 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:49 smithi062 ceph-mon[24799]: pgmap v30: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:22:49.607 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:49 smithi139 ceph-mon[25719]: pgmap v30: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:22:51.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:22:50 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:22:50] "GET /metrics HTTP/1.1" 200 38314 "" "Prometheus/2.33.4" 2024-03-24T11:22:51.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:51 smithi139 ceph-mon[25719]: Upgrade: Updating mgr.x 2024-03-24T11:22:51.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:51 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:51.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:51 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T11:22:51.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:51 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T11:22:51.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:51 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:51.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:51 smithi139 ceph-mon[25719]: Deploying daemon mgr.x on smithi139 2024-03-24T11:22:51.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:51 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:51.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:51 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:51.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:51 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:51.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:51 smithi139 ceph-mon[25719]: pgmap v31: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:22:51.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:51 smithi062 ceph-mon[29666]: Upgrade: Updating mgr.x 2024-03-24T11:22:51.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:51 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:51.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:51 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T11:22:51.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:51 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T11:22:51.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:51 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:51.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:51 smithi062 ceph-mon[29666]: Deploying daemon mgr.x on smithi139 2024-03-24T11:22:51.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:51 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:51.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:51 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:51.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:51 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:51.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:51 smithi062 ceph-mon[29666]: pgmap v31: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:22:51.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:51 smithi062 ceph-mon[24799]: Upgrade: Updating mgr.x 2024-03-24T11:22:51.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:51 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:51.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:51 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T11:22:51.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:51 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T11:22:51.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:51 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:51.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:51 smithi062 ceph-mon[24799]: Deploying daemon mgr.x on smithi139 2024-03-24T11:22:51.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:51 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:51.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:51 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:51.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:51 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:51.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:51 smithi062 ceph-mon[24799]: pgmap v31: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:22:53.903 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:53 smithi139 systemd[1]: Stopping Ceph mgr.x for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:22:54.189 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:53 smithi139 ceph-mon[25719]: pgmap v32: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:22:54.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:53 smithi062 ceph-mon[24799]: pgmap v32: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:22:54.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:53 smithi062 ceph-mon[29666]: pgmap v32: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:22:54.475 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:54 smithi139 podman[67527]: 2024-03-24 11:22:54.188674585 +0000 UTC m=+0.143374253 container died 9079b49d78414e4cd7622dc086e392dd568de76ebe04ae61ef70378aa2f41839 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., GIT_BRANCH=HEAD, io.openshift.expose-services=, RELEASE=HEAD, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, io.buildah.version=1.19.8, io.k8s.display-name=CentOS Stream 8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, version=8, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, vcs-type=git, CEPH_POINT_RELEASE=-17.2.0, ceph=True, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, maintainer=Guillaume Abrioux , vendor=Red Hat, Inc., com.redhat.license_terms=https://centos.org/legal/licensing-policy/, io.openshift.tags=base centos centos-stream, GIT_REPO=https://github.com/ceph/ceph-container.git, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, build-date=2022-05-03T08:36:31.336870, name=centos-stream, release=754, architecture=x86_64, distribution-scope=public, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, GIT_CLEAN=True, com.redhat.component=centos-stream-container) 2024-03-24T11:22:54.799 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:54 smithi139 podman[67527]: 2024-03-24 11:22:54.557479236 +0000 UTC m=+0.512178909 container cleanup 9079b49d78414e4cd7622dc086e392dd568de76ebe04ae61ef70378aa2f41839 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, com.redhat.component=centos-stream-container, architecture=x86_64, GIT_REPO=https://github.com/ceph/ceph-container.git, io.openshift.expose-services=, io.openshift.tags=base centos centos-stream, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, vcs-type=git, CEPH_POINT_RELEASE=-17.2.0, GIT_CLEAN=True, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, version=8, maintainer=Guillaume Abrioux , url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, vendor=Red Hat, Inc., description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, RELEASE=HEAD, io.k8s.display-name=CentOS Stream 8, release=754, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, ceph=True, name=centos-stream, io.buildah.version=1.19.8, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, build-date=2022-05-03T08:36:31.336870, distribution-scope=public, GIT_BRANCH=HEAD) 2024-03-24T11:22:54.800 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:54 smithi139 bash[67527]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x 2024-03-24T11:22:55.108 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:54 smithi139 podman[67541]: 2024-03-24 11:22:54.80008392 +0000 UTC m=+0.613308173 container remove 9079b49d78414e4cd7622dc086e392dd568de76ebe04ae61ef70378aa2f41839 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x, vendor=Red Hat, Inc., com.redhat.component=centos-stream-container, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, RELEASE=HEAD, name=centos-stream, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, io.buildah.version=1.19.8, version=8, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, io.openshift.expose-services=, vcs-type=git, architecture=x86_64, CEPH_POINT_RELEASE=-17.2.0, GIT_CLEAN=True, build-date=2022-05-03T08:36:31.336870, io.openshift.tags=base centos centos-stream, distribution-scope=public, io.k8s.display-name=CentOS Stream 8, maintainer=Guillaume Abrioux , release=754, GIT_REPO=https://github.com/ceph/ceph-container.git, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, GIT_BRANCH=HEAD, ceph=True) 2024-03-24T11:22:55.108 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:54 smithi139 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mgr.x.service: Main process exited, code=exited, status=143/n/a 2024-03-24T11:22:55.108 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:54 smithi139 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mgr.x.service: Failed with result 'exit-code'. 2024-03-24T11:22:55.108 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:54 smithi139 systemd[1]: Stopped Ceph mgr.x for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:22:55.108 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:54 smithi139 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mgr.x.service: Consumed 7.606s CPU time. 2024-03-24T11:22:55.430 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:55 smithi139 systemd[1]: Starting Ceph mgr.x for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:22:55.431 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:55 smithi139 podman[67674]: 2024-03-24 11:22:55.239995767 +0000 UTC m=+0.017000361 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:22:55.431 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:55 smithi139 podman[67674]: 2024-03-24 11:22:55.351656771 +0000 UTC m=+0.128661419 container create 7a48bdfa3fc5870ef4fd9567866c2e4672b22b5ba5f31992a6fb8a3cb3b2c0ee (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-24T11:22:55.858 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:55 smithi139 podman[67674]: 2024-03-24 11:22:55.554095592 +0000 UTC m=+0.331100177 container init 7a48bdfa3fc5870ef4fd9567866c2e4672b22b5ba5f31992a6fb8a3cb3b2c0ee (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240307) 2024-03-24T11:22:55.858 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:55 smithi139 podman[67674]: 2024-03-24 11:22:55.556605108 +0000 UTC m=+0.333609690 container start 7a48bdfa3fc5870ef4fd9567866c2e4672b22b5ba5f31992a6fb8a3cb3b2c0ee (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-db0330b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-24T11:22:55.858 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:55 smithi139 ceph-mgr[67693]: -- 172.21.15.139:0/1231716294 <== mon.2 v2:172.21.15.139:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b0ef2061a0 con 0x55b0ef1dc800 2024-03-24T11:22:55.858 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:55 smithi139 bash[67674]: 7a48bdfa3fc5870ef4fd9567866c2e4672b22b5ba5f31992a6fb8a3cb3b2c0ee 2024-03-24T11:22:55.858 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:55 smithi139 systemd[1]: Started Ceph mgr.x for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:22:56.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:55 smithi062 ceph-mon[29666]: pgmap v33: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:22:56.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:55 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:56.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:55 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:56.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:55 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:56.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:55 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:56.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:55 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:56.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:55 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:56.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:55 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:56.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:55 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]: dispatch 2024-03-24T11:22:56.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:55 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]': finished 2024-03-24T11:22:56.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:55 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:56.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:55 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:56.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:55 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:56.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:55 smithi062 ceph-mon[24799]: pgmap v33: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:22:56.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:55 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:56.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:55 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:56.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:55 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:56.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:55 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:56.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:55 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:56.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:55 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:56.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:55 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:56.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:55 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]: dispatch 2024-03-24T11:22:56.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:55 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]': finished 2024-03-24T11:22:56.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:55 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:56.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:55 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:56.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:55 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:56.232 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[67689]: 2024-03-24T11:22:56.125+0000 7f404345e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-24T11:22:56.232 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[67689]: 2024-03-24T11:22:56.231+0000 7f404345e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-24T11:22:56.232 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:55 smithi139 ceph-mon[25719]: pgmap v33: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:22:56.232 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:55 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:56.232 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:55 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:56.233 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:55 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:56.233 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:55 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:56.233 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:55 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:56.233 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:55 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:56.233 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:55 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:56.233 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:55 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]: dispatch 2024-03-24T11:22:56.233 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:55 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]': finished 2024-03-24T11:22:56.233 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:55 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:56.233 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:55 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:56.233 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:55 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:57.173 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[67689]: 2024-03-24T11:22:56.917+0000 7f404345e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-24T11:22:57.173 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[67689]: 2024-03-24T11:22:57.017+0000 7f404345e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-24T11:22:57.174 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:57.174 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:57.174 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:57.174 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:57.175 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:57.175 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:57.175 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:57.175 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:57.175 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:57.175 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:57.175 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:57.175 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:57.175 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:57.175 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:57.175 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:57.175 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:57.175 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:57.176 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:57.176 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:57.176 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:57.176 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:57.176 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:57.176 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:57.176 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:57.176 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:57.176 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:57.176 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:57.176 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:57.176 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:57.176 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:57.176 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:57.177 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:57.177 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:57.177 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:57.177 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:57.177 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:57.177 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:57.177 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:57.177 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:57.177 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:57.177 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:57.177 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:57.177 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:57.177 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:57.177 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:57.178 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:57.178 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:57.178 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:57.178 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:57.178 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:57.178 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:57.178 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:57.178 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:57.178 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:57.178 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:57.178 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:57.178 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:57.178 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:57.178 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:57.179 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:57.179 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:57.179 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:57.179 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:57.179 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:57.179 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:57.179 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:57.179 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:57.179 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:57.179 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:56 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:57.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:57.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:57.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:57.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:57.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:57.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:57.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:57.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:57.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:57.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:57.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:57.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:57.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:57.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:57.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:57.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:57.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:57.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:57.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:57.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:57.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:57.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:57.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:57.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:57.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:57.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:57.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:57.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:57.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:57.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:57.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:57.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:57.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:57.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:57.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:57.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:57.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:57.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:57.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:57.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:57.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:57.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:57.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:57.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:57.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:57.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:57.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:57.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:57.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:57.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:57.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:57.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:57.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:57.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:57.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:57.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:57.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:57.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:57.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:57.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:57.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:57.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:57.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:57.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:57.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:57.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:57.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:57.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:57.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:57.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:57.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:57.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:57.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:57.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:57.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:57.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:57.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:57.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:57.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:57.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:57.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:57.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:57.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:57.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:57.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:57.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:57.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:57.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:57.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:57.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:57.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:57.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:57.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:57.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:57.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:57.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:57.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:57.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:57.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:57.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:57.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:57.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:57.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:57.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:57.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:57.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:57.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:57.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:57.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:57.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:57.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:57.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:57.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:57.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:57.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:57.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:57.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:57.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:57.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:57.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:57.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:57.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:57.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:57.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:57.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:57.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:57.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:57.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:57.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:57.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:57.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:57.238 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:57.238 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:57.238 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:57.238 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:57.238 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:57.238 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:57.238 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:56 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:57.512 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[67689]: 2024-03-24T11:22:57.172+0000 7f404345e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-24T11:22:57.512 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[67689]: 2024-03-24T11:22:57.285+0000 7f404345e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-24T11:22:57.512 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[67689]: 2024-03-24T11:22:57.374+0000 7f404345e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-24T11:22:57.790 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[67689]: 2024-03-24T11:22:57.672+0000 7f404345e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-24T11:22:57.790 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[67689]: 2024-03-24T11:22:57.789+0000 7f404345e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-24T11:22:58.066 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[67689]: 2024-03-24T11:22:57.882+0000 7f404345e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-24T11:22:58.066 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[67689]: 2024-03-24T11:22:57.986+0000 7f404345e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-24T11:22:58.067 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:58.067 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:58.067 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:58.067 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:58.067 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:58.067 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:58.067 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:58.067 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:58.067 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:58.067 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:58.067 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:58.067 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:58.068 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:58.068 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:58.068 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:58.068 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:58.068 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: pgmap v34: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:22:58.068 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:58.068 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:58.068 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:58.068 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:58.068 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:58.068 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:58.068 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:58.068 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:58.068 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:58.068 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:22:58.069 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:58.069 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:58.069 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:58.069 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:58.069 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:58.069 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:58.069 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:58.069 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:58.070 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:58.070 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:58.070 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:58.070 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:58.070 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:58.070 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:58.070 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:58.070 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:58.070 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:58.070 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:58.070 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:58.071 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:58.071 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:58.071 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:58.071 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:58.071 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:58.071 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:58.071 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:58.071 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:58.071 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:58.071 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:58.071 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:58.071 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:58.071 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:58.071 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:58.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:58.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:58.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:58.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:58.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:58.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:58.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:58.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:58.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:58.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:58.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:58.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:58.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:58.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:58.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:58.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:58.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:58.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:58.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:57 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:58.358 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[67689]: 2024-03-24T11:22:58.181+0000 7f404345e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-24T11:22:58.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:58.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:58.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:58.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:58.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:58.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:58.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:58.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:58.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:58.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:58.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:58.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:58.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:58.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:58.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:58.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:58.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: pgmap v34: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:22:58.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:58.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:58.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:58.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:58.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:58.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:58.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:58.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:58.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:58.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:22:58.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:58.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:58.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:58.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:58.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:58.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:58.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:58.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:58.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:58.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:58.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:58.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:58.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:58.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:58.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:58.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:58.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:58.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:58.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:58.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:58.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:58.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:58.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: pgmap v34: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:22:58.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:58.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:58.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:58.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:58.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:58.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:58.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:58.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:58.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:58.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:22:58.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:58.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:58.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:58.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:58.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:58.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:58.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:58.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:58.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:58.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:58.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:58.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:58.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:58.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:58.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:58.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:58.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:58.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:58.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:58.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:58.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:58.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:58.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:58.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:58.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:58.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:58.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:58.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:58.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:58.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:58.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:58.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:58.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:58.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:58.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:58.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:58.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:58.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:58.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:58.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:58.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:58.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:58.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:58.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:58.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:58.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:58.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:58.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:58.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:58.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:58.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:58.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:58.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:58.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:58.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:58.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:58.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:58.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:58.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:58.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:58.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:58.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:58.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:58.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:58.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:58.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:58.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:58.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:58.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:58.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:58.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:58.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:58.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:58.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:58.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:58.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:58.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:58.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:58.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:58.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:58.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:58.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:58.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:58.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:58.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:58.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:58.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:58.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:58.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:58.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:58.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:58.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:58.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:58.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:58.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:58.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:58.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:58.490 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:57 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:58.981 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:22:58.659Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"smithi062:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"172.21.15.139:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-24T11:22:59.280 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:59.280 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:59.280 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:59.280 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:59.280 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:59.280 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:59.280 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:59.280 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:59.280 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:59.280 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:59.281 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:59.281 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:59.281 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:59.281 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:59.281 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:59.281 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:59.281 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:59.281 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:59.281 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:59.281 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:59.281 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:59.281 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:59.281 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:59.281 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:59.282 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:59.282 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:59.282 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:59.282 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:59.282 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:59.282 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:59.282 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:59.282 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:59.282 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:59.282 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:59.282 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:59.282 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:59.282 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:59.282 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:59.283 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:59.283 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:59.283 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:59.283 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:59.283 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:59.283 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:59.283 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:59.283 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:59.283 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:59.283 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:59.283 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:59.283 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:59.283 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:59.283 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:59.283 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:59.284 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:59.284 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:59.284 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:59.284 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:59.284 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:59.284 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:59.284 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:59.284 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:59.284 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:59.284 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:59.284 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:59.284 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:59.284 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:59.284 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:59.285 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:59.285 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:59.285 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:59.285 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:59.285 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:58 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:59.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:59.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:59.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:59.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:59.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:59.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:59.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:59.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:59.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:59.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:59.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:59.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:59.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:59.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:59.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:59.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:59.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:59.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:59.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:59.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:59.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:59.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:59.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:59.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:59.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:59.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:59.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:59.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:59.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:59.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:59.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:59.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:59.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:59.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:59.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:59.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:59.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:59.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:59.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:59.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:59.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:59.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:59.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:59.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:59.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:59.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:59.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:59.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:59.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:59.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:59.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:59.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:59.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:59.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:59.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:59.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:59.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:59.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:59.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:59.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:59.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:59.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:59.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:59.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:59.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:59.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:59.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:59.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:59.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:59.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:59.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:59.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:59.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:59.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:59.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:59.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:59.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:59.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:59.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:59.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:59.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:59.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:59.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:59.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:59.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:59.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:59.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:59.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:59.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:22:59.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:59.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:59.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:59.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:22:59.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:59.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:59.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:59.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:59.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:59.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:59.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:59.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:59.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:59.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:59.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:59.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:59.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:59.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:59.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:59.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:59.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:59.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:59.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:59.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:59.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:59.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:59.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:59.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:59.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:59.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:59.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:59.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:59.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:59.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:59.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:59.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:59.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:59.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:59.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:59.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:59.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:59.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:59.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:59.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:59.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:59.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:59.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:59.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:59.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:59.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:59.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:22:59.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:22:59.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:22:59.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:22:59.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:58 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:22:59.567 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[67689]: 2024-03-24T11:22:59.277+0000 7f404345e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-24T11:22:59.567 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[67689]: 2024-03-24T11:22:59.416+0000 7f404345e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-24T11:22:59.858 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[67689]: 2024-03-24T11:22:59.781+0000 7f404345e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-24T11:23:00.143 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[67689]: 2024-03-24T11:22:59.878+0000 7f404345e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-24T11:23:00.143 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[67689]: 2024-03-24T11:23:00.094+0000 7f404345e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-24T11:23:00.143 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:00.143 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:00.144 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:00.144 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:00.144 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:00.144 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:00.144 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:00.144 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:00.144 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:00.144 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:00.144 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:00.144 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:00.144 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:00.144 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:00.144 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: pgmap v35: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:23:00.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:00.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:00.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:00.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:00.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:00.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:00.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:00.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:00.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:00.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:00.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:00.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:00.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:00.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:00.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:00.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:00.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:00.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:00.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:00.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:00.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:00.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:00.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:00.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:00.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:00.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:00.147 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:00.147 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:00.147 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:00.147 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:00.147 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:00.147 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:00.147 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:00.147 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:00.147 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:00.147 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:00.147 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:00.147 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:00.147 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:00.147 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:00.147 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:00.148 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:00.148 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:00.148 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:00.148 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:00.148 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:00.148 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:00.148 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:00.148 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:00.148 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:00.148 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:00.148 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:00.149 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:00.149 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:00.149 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:00.149 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:00.149 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:00.149 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:00.149 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:00.149 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:00.149 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:00.149 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:00.149 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:22:59 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:00.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:00.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:00.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:00.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:00.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:00.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:00.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:00.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:00.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:00.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:00.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:00.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:00.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:00.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:00.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: pgmap v35: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:23:00.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:00.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:00.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:00.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:00.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:00.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:00.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:00.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:00.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:00.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:00.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:00.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:00.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:00.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:00.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:00.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:00.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:00.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:00.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:00.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:00.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:00.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:00.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:00.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:00.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:00.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:00.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:00.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:00.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:00.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:00.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:00.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:00.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: pgmap v35: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:23:00.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:00.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:00.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:00.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:00.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:00.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:00.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:00.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:00.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:00.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:00.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:00.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:00.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:00.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:00.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:00.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:00.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:00.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:00.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:00.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:00.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:00.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:00.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:00.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:00.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:00.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:00.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:00.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:00.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:00.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:00.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:00.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:00.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:00.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:00.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:00.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:00.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:00.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:00.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:00.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:00.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:00.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:00.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:00.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:00.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:00.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:00.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:00.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:00.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:00.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:00.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:00.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:00.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:00.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:00.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:00.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:00.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:00.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:00.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:00.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:00.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:00.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:00.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:00.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:00.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:00.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:00.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:00.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:00.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:00.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:00.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:00.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:00.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:00.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:00.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:00.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:00.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:00.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:00.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:00.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:00.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:00.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:00.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:00.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:00.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:00.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:00.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:00.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:00.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:00.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:00.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:00.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:00.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:00.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:00.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:00.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:00.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:00.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:00.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:00.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:00.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:00.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:00.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:00.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:00.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:00.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:00.490 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:00.490 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:22:59 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:00.490 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[67689]: 2024-03-24T11:23:00.295+0000 7f404345e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-24T11:23:00.857 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[67689]: 2024-03-24T11:23:00.610+0000 7f404345e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-24T11:23:01.144 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[67689]: 2024-03-24T11:23:00.887+0000 7f404345e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-24T11:23:01.144 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[67689]: 2024-03-24T11:23:01.143+0000 7f404345e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-24T11:23:01.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:01.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:01.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:01.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:01.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:01.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:01.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:01.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:01.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:01.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:01.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:01.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:01.145 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:01.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:01.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:01.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:01.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:01.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:01.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:01.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:01.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:01.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:01.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:01.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:01.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:01.146 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:01.147 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:01.147 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:01.147 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:01.147 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:01.147 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:01.147 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:01.147 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:01.147 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:01.147 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:01.147 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:01.147 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:01.147 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:01.147 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:01.147 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:01.147 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:01.148 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:01.148 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:01.148 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:01.148 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:01.148 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:01.148 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:01.148 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:01.148 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:01.148 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:01.148 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:01.148 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:01.148 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:01.148 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:01.148 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:01.149 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:01.149 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:01.149 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:01.149 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:01.149 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:01.149 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:01.149 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:01.149 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:01.149 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:01.149 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:01.149 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:01.149 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:01.150 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:01.150 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:01.150 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:01.150 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:01.150 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:01.150 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:01.150 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:01.150 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:01.150 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:01.150 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:01.150 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:01.150 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:01.150 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:01.150 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:01.151 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:00 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:01.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:23:00] "GET /metrics HTTP/1.1" 200 38314 "" "Prometheus/2.33.4" 2024-03-24T11:23:01.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:01.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:01.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:01.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:01.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:01.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:01.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:01.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:01.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:01.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:01.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:01.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:01.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:01.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:01.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:01.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:01.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:01.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:01.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:01.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:01.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:01.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:01.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:01.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:01.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:01.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:01.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:01.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:01.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:01.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:01.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:01.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:01.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:01.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:01.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:01.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:01.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:01.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:01.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:01.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:01.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:01.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:01.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:01.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:01.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:01.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:01.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:01.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:01.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:01.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:01.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:01.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:01.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:01.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:01.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:01.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:01.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:01.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:01.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:01.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:01.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:01.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:01.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:01.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:01.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:01.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:01.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:01.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:01.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:01.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:01.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:01.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:01.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:01.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:01.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:01.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:01.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:01.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:01.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:01.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:01.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:01.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:01.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:01.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:01.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:01.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:01.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:01.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:01.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:01.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:01.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:01.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:01.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:01.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:01.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:01.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:01.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:01.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:01.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:01.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:01.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:01.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:01.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:01.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:01.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:01.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:01.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:01.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:01.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:01.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:01.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:01.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:01.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:01.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:01.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:01.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:01.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:01.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:01.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:01.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:01.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:01.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:01.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:01.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:01.238 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:01.238 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:01.238 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:01.238 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:01.238 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:01.238 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:01.238 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:01.238 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:01.238 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:01.238 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:01.239 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:01.239 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:01.239 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:01.239 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:01.239 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:01.239 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:01.239 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:01.239 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:01.239 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:01.239 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:01.239 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:01.240 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:01.240 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:01.240 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:01.240 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:01.240 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:01.240 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:01.240 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:01.240 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:01.240 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:01.240 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:01.240 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:01.240 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:01.240 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:01.240 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:01.240 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:01.241 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:01.241 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:01.241 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:01.241 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:00 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:02.001 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:02.001 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:02.001 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:02.001 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:02.001 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:02.001 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:02.001 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:02.001 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:02.001 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:02.001 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:02.002 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:02.002 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:02.002 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:02.002 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:02.002 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:02.002 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:02.002 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: pgmap v36: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:23:02.002 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:02.002 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:02.002 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:02.002 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:02.002 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:02.002 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:02.003 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:02.003 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:02.003 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:02.003 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:02.003 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:02.003 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:02.003 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:02.003 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:02.003 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:02.003 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:02.004 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:02.004 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:02.004 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:02.004 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:02.004 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:02.004 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:02.004 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:02.004 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:02.004 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:02.005 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:02.005 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:02.005 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:02.005 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:02.005 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:02.005 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:02.005 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:02.005 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:02.005 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:02.005 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:02.005 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:02.005 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:02.005 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:02.005 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:02.006 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:02.006 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:02.006 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:02.006 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:02.006 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:02.006 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:02.006 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:02.006 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:02.006 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:02.006 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:02.006 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:02.006 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:02.006 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:02.007 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:02.007 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:02.007 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:02.007 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:02.007 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:02.007 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:02.007 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:02.007 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:02.007 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:02.007 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:02.007 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:02.007 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:02.007 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:02.007 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:02.008 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:02.008 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:01 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:02.357 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[67689]: 2024-03-24T11:23:02.024+0000 7f404345e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-24T11:23:02.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:02.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:02.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:02.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:02.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:02.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:02.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:02.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:02.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:02.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:02.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:02.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:02.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:02.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:02.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:02.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:02.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: pgmap v36: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:23:02.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:02.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:02.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:02.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:02.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:02.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:02.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:02.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:02.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:02.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:02.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:02.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:02.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:02.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:02.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:02.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:02.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:02.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:02.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:02.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:02.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:02.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:02.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:02.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:02.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:02.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:02.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:02.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:02.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:02.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:02.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:02.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:02.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: pgmap v36: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:23:02.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:02.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:02.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:02.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:02.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:02.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:02.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:02.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:02.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:02.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:02.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:02.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:02.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:02.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:02.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:02.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:02.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:02.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:02.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:02.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:02.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:02.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:02.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:02.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:02.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:02.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:02.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:02.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:02.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:02.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:02.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:02.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:02.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:02.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:02.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:02.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:02.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:02.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:02.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:02.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:02.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:02.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:02.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:02.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:02.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:02.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:02.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:02.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:02.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:02.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:02.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:02.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:02.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:02.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:02.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:02.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:02.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:02.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:02.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:02.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:02.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:02.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:02.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:02.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:02.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:02.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:02.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:02.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:02.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:02.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:02.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:02.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:02.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:02.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:02.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:02.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:02.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:02.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:02.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:02.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:02.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:02.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:02.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:02.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:02.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:02.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:02.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:02.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:02.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:02.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:02.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:02.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:02.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:02.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:02.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:02.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:02.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:02.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:02.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:02.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:02.490 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:02.490 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:02.490 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:02.490 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:02.490 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:02.490 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:02.490 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:02.490 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:02.490 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:02.490 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:02.490 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:02.490 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:02.490 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:02.491 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:02.491 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:02.491 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:02.491 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:02.491 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:02.491 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:02.491 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:01 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:02.779 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[67689]: 2024-03-24T11:23:02.665+0000 7f404345e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-24T11:23:03.030 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[67689]: 2024-03-24T11:23:02.778+0000 7f404345e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-24T11:23:03.030 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[67689]: 2024-03-24T11:23:03.014+0000 7f404345e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-24T11:23:03.030 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:03.031 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:03.031 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:03.031 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:03.031 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:03.031 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:03.031 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:03.031 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:03.031 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:03.031 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:03.031 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:03.031 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:03.031 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:03.031 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:03.032 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:03.032 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:03.032 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:03.032 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:03.032 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:03.032 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:03.032 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:03.032 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:03.032 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:03.032 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:03.032 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:03.032 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:03.032 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:03.032 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:03.032 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:03.033 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:03.033 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:03.033 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:03.033 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:23:03.033 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:03.033 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:03.033 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:03.034 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:03.034 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:03.034 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:03.034 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:03.034 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:03.034 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:03.034 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:03.034 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:03.034 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:03.034 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:03.034 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:03.034 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:03.035 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:03.035 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:03.035 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:03.035 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:03.035 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:03.035 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:03.035 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:03.035 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:03.035 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:03.035 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:03.036 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:03.036 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:03.036 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:03.036 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:03.036 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:03.036 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:03.036 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:03.036 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:03.036 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:03.036 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:03.036 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:03.036 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:03.036 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:03.036 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:03.037 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:02 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:03.307 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[67689]: [24/Mar/2024:11:23:03] ENGINE Bus STARTING 2024-03-24T11:23:03.307 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[67689]: CherryPy Checker: 2024-03-24T11:23:03.307 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[67689]: The Application mounted at '' has an empty config. 2024-03-24T11:23:03.307 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[67689]: 2024-03-24T11:23:03.307 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[67689]: [24/Mar/2024:11:23:03] ENGINE Serving on http://:::9283 2024-03-24T11:23:03.307 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[67689]: [24/Mar/2024:11:23:03] ENGINE Bus STARTED 2024-03-24T11:23:03.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:03.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:03.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:03.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:03.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:03.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:03.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:03.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:03.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:03.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:03.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:03.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:03.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:03.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:03.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:03.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:03.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:03.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:03.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:03.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:03.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:03.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:03.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:03.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:03.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:03.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:03.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:03.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:03.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:03.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:03.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:03.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:03.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:23:03.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:03.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:03.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:03.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:03.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:03.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:03.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:03.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:03.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:03.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:03.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:03.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:03.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:03.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:03.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:03.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:03.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:03.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:03.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:03.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:03.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:03.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:03.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:03.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:03.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:03.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:03.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:03.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:03.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:03.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:03.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:03.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:03.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:23:03.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:03.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:03.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:03.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:03.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:03.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:03.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:03.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:03.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:03.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:03.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:03.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:03.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:03.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:03.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:03.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:03.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:03.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:03.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:03.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:03.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:03.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:03.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:03.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:03.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:03.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:03.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:03.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:03.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:03.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:03.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:03.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:03.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:03.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:03.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:03.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:03.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:03.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:03.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:03.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:03.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:03.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:03.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:03.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:03.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:03.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:03.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:03.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:03.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:03.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:03.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:03.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:03.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:03.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:03.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:03.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:03.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:03.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:03.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:03.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:03.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:03.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:03.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:03.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:03.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:03.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:03.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:03.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:03.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:03.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:03.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:03.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:03.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:03.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:03.490 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:03.490 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:03.490 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:03.490 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:03.490 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:03.490 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:02 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:04.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:04.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:04.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:04.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:04.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:04.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:04.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:04.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:04.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:04.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:04.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:04.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:04.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:04.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:04.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:04.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:04.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: pgmap v37: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:23:04.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:04.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:04.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:04.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:04.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:04.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:04.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:04.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: Standby manager daemon x restarted 2024-03-24T11:23:04.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: Standby manager daemon x started 2024-03-24T11:23:04.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: from='mgr.? 172.21.15.139:0/3060458306' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-24T11:23:04.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: from='mgr.? 172.21.15.139:0/3060458306' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T11:23:04.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: from='mgr.? 172.21.15.139:0/3060458306' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-24T11:23:04.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: from='mgr.? 172.21.15.139:0/3060458306' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T11:23:04.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:04.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:04.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:04.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:04.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:04.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:04.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:04.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:04.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:04.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:04.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:04.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:04.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:04.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:04.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:04.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:04.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:04.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:04.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:04.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:04.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:04.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:04.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:04.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:04.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:04.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:04.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:04.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:04.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:04.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:04.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:04.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:04.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:04.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:04.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:04.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:04.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:04.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:04.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:03 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:04.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:04.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:04.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:04.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:04.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:04.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:04.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:04.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:04.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:04.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:04.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:04.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:04.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:04.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:04.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:04.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:04.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: pgmap v37: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:23:04.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:04.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:04.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:04.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:04.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:04.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:04.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:04.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: Standby manager daemon x restarted 2024-03-24T11:23:04.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: Standby manager daemon x started 2024-03-24T11:23:04.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: from='mgr.? 172.21.15.139:0/3060458306' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-24T11:23:04.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: from='mgr.? 172.21.15.139:0/3060458306' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T11:23:04.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: from='mgr.? 172.21.15.139:0/3060458306' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-24T11:23:04.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: from='mgr.? 172.21.15.139:0/3060458306' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T11:23:04.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:04.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:04.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:04.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:04.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:04.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:04.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:04.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:04.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:04.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:04.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:04.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:04.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:04.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:04.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:04.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:04.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:04.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:04.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:04.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: pgmap v37: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:23:04.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:04.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:04.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:04.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:04.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:04.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:04.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:04.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: Standby manager daemon x restarted 2024-03-24T11:23:04.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: Standby manager daemon x started 2024-03-24T11:23:04.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: from='mgr.? 172.21.15.139:0/3060458306' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-24T11:23:04.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: from='mgr.? 172.21.15.139:0/3060458306' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T11:23:04.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: from='mgr.? 172.21.15.139:0/3060458306' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-24T11:23:04.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: from='mgr.? 172.21.15.139:0/3060458306' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T11:23:04.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:04.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:04.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:04.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:04.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:04.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:04.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:04.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:04.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:04.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:04.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:04.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:04.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:04.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:04.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:04.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:04.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:04.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:04.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:04.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:04.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:04.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:04.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:04.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:04.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:04.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:04.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:04.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:04.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:04.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:04.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:04.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:04.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:04.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:04.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:04.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:04.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:04.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:04.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:04.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:04.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:04.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:04.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:04.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:04.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:04.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:04.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:04.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:04.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:04.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:04.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:04.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:04.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:04.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:04.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:04.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:04.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:04.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:04.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:04.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:04.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:04.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:04.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:04.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:04.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:04.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:04.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:04.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:04.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:04.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:04.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:04.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:04.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:04.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:04.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:03 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:05.161 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:05.161 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:05.161 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:05.161 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:05.161 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:05.161 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:05.162 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:05.162 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:05.162 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:05.162 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:05.162 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: mgrmap e36: y(active, since 62s), standbys: x 2024-03-24T11:23:05.162 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:05.162 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:05.162 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:05.162 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:05.162 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:05.162 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:05.162 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:05.162 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:05.162 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:05.163 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:05.163 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:05.163 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:05.163 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:05.163 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:05.163 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:05.163 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:05.163 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:05.163 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:05.163 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:05.163 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:05.164 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:05.164 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:05.164 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:05.164 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:05.164 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:05.164 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:05.164 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:05.164 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:05.164 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:05.164 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:05.164 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:05.164 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:05.165 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:05.165 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:05.165 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:05.165 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:05.165 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:05.165 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:05.165 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:05.165 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:05.165 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:05.165 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:05.165 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:05.165 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:05.165 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:05.165 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:05.166 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:05.166 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:05.166 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:05.166 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:05.166 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:05.166 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:05 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:05.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:05.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:05.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:05.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:05.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:05.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:05.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:05.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:05.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:05.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:05.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: mgrmap e36: y(active, since 62s), standbys: x 2024-03-24T11:23:05.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:05.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:05.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:05.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:05.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:05.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:05.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:05.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:05.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:05.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:05.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:05.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:05.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:05.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:05.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:05.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:05.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:05.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:05.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:05.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:05.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:05.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:05.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:05.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:05.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:05.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:05.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:05.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:05.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: Upgrade: 1 mgr daemon(s) are 17.2.0 != target 19.0.0-1667-gdb0330b1 2024-03-24T11:23:05.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:05.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: mgrmap e36: y(active, since 62s), standbys: x 2024-03-24T11:23:05.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:05.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:05.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:05.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:05.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:05.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:05.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:05.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:05.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:05.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:05.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:05.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:05.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:05.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:05.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:05.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:05.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:05.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:05.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:05.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:05.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:05.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:05.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:05.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:05.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:05.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:05.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:05.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:05.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:05.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:05.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:05.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:05.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:05.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:05.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:05.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:05.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:05.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:05.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:05.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:05.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:05.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:05.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:05.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:05.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:05.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:05.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:05.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:05.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:05.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:05.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:05.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:05.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:05.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:05.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:05.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:05.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:05.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:05.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:05.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:05.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:05.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:05.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:05.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:05.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:05.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:05.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:05.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:05.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:05.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:05.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:05.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:05.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:05.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:05.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:05.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:05.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:05.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:05.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:05.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:05.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:05.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:05.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:05.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:05 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:06.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:06.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:06.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:06.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:06.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:06.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:06.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:06.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:06.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: pgmap v38: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:23:06.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:06.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:06.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:06.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:06.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:06.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:06.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:06.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:06.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:06.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:06.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:06.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:06.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:06.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:06.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:06.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:06.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:06.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:06.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:06.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:06.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:06.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:06.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:06.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:06.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:06.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:06.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:06.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:06.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:06.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:06.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:06.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:06.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:06.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:06.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:06.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:06.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:06.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:06.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:06.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:06.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:06.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:06.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:06.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:06.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:06.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:06.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:06 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:06.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:06.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:06.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:06.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:06.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:06.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:06.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:06.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:06.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: pgmap v38: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:23:06.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:06.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:06.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:06.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:06.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:06.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:06.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:06.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:06.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:06.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:06.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:06.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:06.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:06.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:06.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:06.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:06.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:06.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:06.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:06.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:06.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:06.476 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:06.476 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:06.476 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:06.476 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:06.476 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:06.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:06.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:06.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:06.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: pgmap v38: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:23:06.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:06.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:06.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:06.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:06.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:06.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:06.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:06.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:06.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:06.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:06.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:06.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:06.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:06.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:06.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:06.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:06.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:06.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:06.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:06.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:06.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:06.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:06.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:06.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:06.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:06.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:06.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:06.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:06.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:06.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:06.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:06.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:06.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:06.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:06.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:06.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:06.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:06.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:06.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:06.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:06.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:06.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:06.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:06.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:06.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:06.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:06.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:06.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:06.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:06.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:06.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:06.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:06.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:06.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:06.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:06.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:06.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:06.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:06.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:06.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:06.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:06.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:06.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:06.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:06.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:06.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:06.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:06.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:06.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:06.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:06.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:06.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:06.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:06 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:07.139 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:07.139 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:07.139 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:07.139 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:07.139 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:07.139 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:07.140 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:07.140 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:07.140 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:07.140 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:07.140 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:07.140 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:07.140 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:07.140 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:07.140 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:07.140 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:07.140 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:07.140 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:07.140 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:07.140 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:07.140 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:07.141 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:07.141 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:07.141 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:07.141 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:07.141 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:07.141 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:07.141 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:07.141 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:07.141 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:07.141 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:07.141 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:07.141 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:07.142 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:07.142 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:07.142 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:07.142 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:07.142 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:07.142 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:07.142 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:07.142 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:07.142 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:07.143 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:07.143 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:07.143 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:07.143 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:07.143 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:07.143 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:07.143 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:07.143 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:07.143 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:07.143 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:07.144 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:07.144 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:07.144 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:07.144 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:07.144 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:07.144 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:07.144 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:07.144 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:07.144 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:07.144 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:07.144 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:07.145 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:07.145 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:07.145 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:07.145 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:07.145 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:07.145 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:07.145 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:07.145 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:07.145 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:07.145 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:07.145 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:07.145 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:07.145 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:07.145 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:07.146 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:07.146 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:07.146 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:07.146 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:07.146 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:07.146 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:07.146 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:07.146 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:07.146 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:07.146 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:07.146 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:07.146 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:07.147 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:07.147 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:07.147 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:07.147 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:07.147 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:07.147 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:07.147 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:07.147 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:07.147 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:07.147 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:07.147 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:07.147 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:07.147 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:07.147 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:07.148 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:07 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:07.308 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:07.308 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:07.308 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:07.309 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:07.309 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:07.309 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:07.309 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:07.309 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:07.309 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:07.309 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:07.309 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:07.309 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:07.309 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:07.309 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:07.309 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:07.309 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:07.309 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:07.309 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:07.310 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:07.310 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:07.310 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:07.310 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:07.310 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:07.310 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:07.310 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:07.310 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:07.310 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:07.310 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:07.310 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:07.310 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:07.310 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:07.310 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:07.310 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:07.311 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:07.311 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:07.311 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:07.311 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:07.311 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:07.311 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:07.311 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:07.311 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:07.311 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:07.311 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:07.311 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:07.311 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:07.311 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:07.311 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:07.312 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:07.312 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:07.312 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:07.312 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:07.312 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:07 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:07.739 INFO:teuthology.orchestra.run.smithi062.stdout:true 2024-03-24T11:23:08.332 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:08.332 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:08.332 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:08.332 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:08.332 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:08.332 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:08.332 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:08.332 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: pgmap v39: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:23:08.333 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:08.333 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:08.333 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:23:08.333 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:08.333 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:08.333 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:08.333 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:08.333 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:08.333 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:08.333 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:08.333 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:08.333 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:08.333 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:08.333 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:08.334 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:08.334 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:08.334 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:08.334 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:08.334 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:08.334 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:08.334 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:08.334 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:08.334 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:08.334 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:08.334 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:08.335 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:08.335 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:08.335 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:08.335 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:08.335 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: pgmap v39: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:23:08.335 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:08.335 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:08.335 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:23:08.335 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:08.335 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:08.335 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:08.335 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:08.335 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:08.335 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:08.335 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:08.336 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:08.336 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:08.336 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:08.336 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:08.336 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:08.336 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:08.336 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:08.336 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:08.336 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:08.336 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:08.336 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:08.336 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:08.337 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:08.337 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:08.337 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:08.337 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:08.337 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:08.337 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:08.337 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:08.337 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:08.337 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:08.337 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:08.337 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:08.337 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:08.337 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:08.337 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:08.338 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:08.338 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:08.338 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:08.338 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:08.338 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:08.338 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:08.338 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:08.338 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:08.338 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:08.338 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:08.338 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:08.338 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:08.338 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:08.339 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:08.339 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:08.339 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:08.339 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:08.339 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:08.339 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:08.339 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:08.339 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:08.339 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:08.339 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:08.339 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:08.339 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:08.340 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:08.340 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:08.340 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:08.340 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:08.340 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:08.340 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:08.340 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:08.340 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:08.340 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:08.340 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:08.340 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:08.340 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:08.340 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:08.340 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:08.341 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:08 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:08.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:08.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:08.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:08.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:08.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:08.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:08.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:08.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: pgmap v39: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:23:08.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:08.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:08.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:23:08.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:08.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:08.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:08.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:08.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:08.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:08.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:08.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:08.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:08.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:08.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:08.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:08.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:08.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:08.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:08.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:08.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:08.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:08.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:08.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:08.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:08.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:08.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:08.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:08.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:08.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:08.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:08.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:08.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:08.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:08.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:08.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:08.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:08.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:08.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:08.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:08.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:08.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:08.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:08.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:08.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:08.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:08.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:08.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:08.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:08.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:08.437 INFO:teuthology.orchestra.run.smithi062.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T11:23:08.437 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi062 smithi062 running 17s ago 12m - - 2024-03-24T11:23:08.437 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi139 smithi139 running 0s ago 12m - - 2024-03-24T11:23:08.437 INFO:teuthology.orchestra.run.smithi062.stdout:alertmanager.a smithi062 *:9093,9094 running (2m) 17s ago 7m 17.7M - 0.23.0 ba2b418f427c 818298d58140 2024-03-24T11:23:08.437 INFO:teuthology.orchestra.run.smithi062.stdout:grafana.a smithi139 *:3000 running (103s) 0s ago 6m 47.8M - 8.3.5 dad864ee21e9 b48167e7171e 2024-03-24T11:23:08.437 INFO:teuthology.orchestra.run.smithi062.stdout:iscsi.foo.smithi062.dggsvk smithi062 running (41s) 17s ago 5m 50.3M - 3.5 e1d6a67b021e 0a98806ec839 2024-03-24T11:23:08.438 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.x smithi139 *:8443,9283,8765 running (28s) 0s ago 11m 412M - 17.2.0 e1d6a67b021e 9079b49d7841 2024-03-24T11:23:08.438 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.y smithi062 *:8443,9283,8765 running (74s) 17s ago 13m 506M - 19.0.0-1667-gdb0330b1 4f325f8440e9 ca1a49028d65 2024-03-24T11:23:08.438 INFO:teuthology.orchestra.run.smithi062.stdout:mon.a smithi062 running (13m) 17s ago 13m 60.3M 2048M 17.2.0 e1d6a67b021e 712f01aa9180 2024-03-24T11:23:08.438 INFO:teuthology.orchestra.run.smithi062.stdout:mon.b smithi139 running (12m) 0s ago 12m 51.5M 2048M 17.2.0 e1d6a67b021e 8945cdb4e6aa 2024-03-24T11:23:08.438 INFO:teuthology.orchestra.run.smithi062.stdout:mon.c smithi062 running (12m) 17s ago 12m 46.4M 2048M 17.2.0 e1d6a67b021e 39110a16ab62 2024-03-24T11:23:08.438 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.a smithi062 *:9100 running (2m) 17s ago 8m 15.8M - 1.3.1 1dbe0e931976 79d5a02a999f 2024-03-24T11:23:08.438 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.b smithi139 *:9100 running (118s) 0s ago 7m 18.0M - 1.3.1 1dbe0e931976 8f9f4608e465 2024-03-24T11:23:08.438 INFO:teuthology.orchestra.run.smithi062.stdout:osd.0 smithi062 running (11m) 17s ago 11m 53.0M 4096M 17.2.0 e1d6a67b021e 8040b30d7170 2024-03-24T11:23:08.438 INFO:teuthology.orchestra.run.smithi062.stdout:osd.1 smithi062 running (11m) 17s ago 11m 54.8M 4096M 17.2.0 e1d6a67b021e e62d22e5dc29 2024-03-24T11:23:08.438 INFO:teuthology.orchestra.run.smithi062.stdout:osd.2 smithi062 running (10m) 17s ago 10m 48.2M 4096M 17.2.0 e1d6a67b021e 714f550103a7 2024-03-24T11:23:08.438 INFO:teuthology.orchestra.run.smithi062.stdout:osd.3 smithi062 running (10m) 17s ago 10m 52.6M 4096M 17.2.0 e1d6a67b021e 1d2868bd5195 2024-03-24T11:23:08.438 INFO:teuthology.orchestra.run.smithi062.stdout:osd.4 smithi139 running (9m) 0s ago 9m 56.9M 4096M 17.2.0 e1d6a67b021e 7568bfa9e74d 2024-03-24T11:23:08.438 INFO:teuthology.orchestra.run.smithi062.stdout:osd.5 smithi139 running (9m) 0s ago 9m 50.3M 4096M 17.2.0 e1d6a67b021e dd0abd599a14 2024-03-24T11:23:08.438 INFO:teuthology.orchestra.run.smithi062.stdout:osd.6 smithi139 running (9m) 0s ago 9m 50.6M 4096M 17.2.0 e1d6a67b021e a4b1289aca69 2024-03-24T11:23:08.439 INFO:teuthology.orchestra.run.smithi062.stdout:osd.7 smithi139 running (8m) 0s ago 8m 52.5M 4096M 17.2.0 e1d6a67b021e bc7fadbc1384 2024-03-24T11:23:08.439 INFO:teuthology.orchestra.run.smithi062.stdout:prometheus.a smithi139 *:9095 running (53s) 0s ago 8m 53.5M - 2.33.4 514e6a882f6e 2ed47f810d33 2024-03-24T11:23:08.439 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi062.sacpds smithi062 *:8000 running (6m) 17s ago 6m 72.4M - 17.2.0 e1d6a67b021e 89f4ac308e98 2024-03-24T11:23:08.439 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi139.yljesh smithi139 *:8000 running (6m) 0s ago 6m 71.2M - 17.2.0 e1d6a67b021e e189bdc12cae 2024-03-24T11:23:08.439 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi062.xvpxnk smithi062 *:80 running (6m) 17s ago 6m 72.0M - 17.2.0 e1d6a67b021e d1aa74adab44 2024-03-24T11:23:08.439 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi139.pbtrmp smithi139 *:80 running (6m) 0s ago 6m 74.6M - 17.2.0 e1d6a67b021e 7f8e336755d7 2024-03-24T11:23:08.810 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:23:08 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:23:08.659Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"smithi062:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"172.21.15.139:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-24T11:23:08.850 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:23:08.851 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": { 2024-03-24T11:23:08.851 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 3 2024-03-24T11:23:08.851 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:23:08.851 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": { 2024-03-24T11:23:08.851 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-24T11:23:08.851 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:23:08.851 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": { 2024-03-24T11:23:08.851 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 8 2024-03-24T11:23:08.851 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:23:08.851 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": {}, 2024-03-24T11:23:08.851 INFO:teuthology.orchestra.run.smithi062.stdout: "rgw": { 2024-03-24T11:23:08.851 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4 2024-03-24T11:23:08.851 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:23:08.852 INFO:teuthology.orchestra.run.smithi062.stdout: "overall": { 2024-03-24T11:23:08.852 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 15, 2024-03-24T11:23:08.852 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-24T11:23:08.852 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-24T11:23:08.852 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:23:09.070 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:09 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:09.071 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:09 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:09.071 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:09 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:09.071 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:09 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:09.071 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:09 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:09.071 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:09 smithi139 ceph-mon[25719]: from='client.15543 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:23:09.071 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:09 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:09.071 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:09 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:09.071 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:09 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:09.071 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:09 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:09.071 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:09 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:09.071 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:09 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:09.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:09 smithi139 ceph-mon[25719]: from='client.15549 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:23:09.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:09 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:09.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:09 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:09.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:09 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:09.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:09 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:09.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:09 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/3304723628' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:09.203 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:23:09.203 INFO:teuthology.orchestra.run.smithi062.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69", 2024-03-24T11:23:09.203 INFO:teuthology.orchestra.run.smithi062.stdout: "in_progress": true, 2024-03-24T11:23:09.203 INFO:teuthology.orchestra.run.smithi062.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T11:23:09.204 INFO:teuthology.orchestra.run.smithi062.stdout: "services_complete": [ 2024-03-24T11:23:09.204 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr" 2024-03-24T11:23:09.204 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-03-24T11:23:09.204 INFO:teuthology.orchestra.run.smithi062.stdout: "progress": "2/23 daemons upgraded", 2024-03-24T11:23:09.204 INFO:teuthology.orchestra.run.smithi062.stdout: "message": "Currently upgrading mgr daemons", 2024-03-24T11:23:09.204 INFO:teuthology.orchestra.run.smithi062.stdout: "is_paused": false 2024-03-24T11:23:09.204 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:23:09.204 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:09 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:09.204 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:09 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:09.204 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:09 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:09.204 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:09 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:09.205 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:09 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:09.205 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:09 smithi062 ceph-mon[24799]: from='client.15543 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:23:09.205 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:09 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:09.205 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:09 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:09.205 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:09 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:09.205 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:09 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:09.205 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:09 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:09.205 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:09 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:09.205 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:09 smithi062 ceph-mon[24799]: from='client.15549 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:23:09.205 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:09 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:09.205 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:09 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:09.205 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:09 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:09.205 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:09 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:09.205 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:09 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/3304723628' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:09.206 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:09 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:09.206 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:09 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:09.206 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:09 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:09.206 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:09 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:09.206 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:09 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:09.206 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:09 smithi062 ceph-mon[29666]: from='client.15543 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:23:09.206 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:09 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:09.206 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:09 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:09.206 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:09 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:09.206 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:09 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:09.206 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:09 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:09.206 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:09 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:09.207 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:09 smithi062 ceph-mon[29666]: from='client.15549 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:23:09.207 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:09 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:09.207 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:09 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:09.207 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:09 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:09.207 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:09 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:09.207 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:09 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/3304723628' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:09.621 INFO:teuthology.orchestra.run.smithi062.stdout:HEALTH_OK 2024-03-24T11:23:10.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:10 smithi139 ceph-mon[25719]: from='client.15555 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:23:10.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:10 smithi139 ceph-mon[25719]: pgmap v40: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:23:10.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:10 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:10.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:10 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:10.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:10 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:10.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:10 smithi139 ceph-mon[25719]: from='client.15567 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:23:10.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:10 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/460774774' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:23:10.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:10 smithi062 ceph-mon[24799]: from='client.15555 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:23:10.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:10 smithi062 ceph-mon[24799]: pgmap v40: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:23:10.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:10 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:10.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:10 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:10.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:10 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:10.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:10 smithi062 ceph-mon[24799]: from='client.15567 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:23:10.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:10 smithi062 ceph-mon[24799]: from='client.? 172.21.15.62:0/460774774' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:23:10.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:10 smithi062 ceph-mon[29666]: from='client.15555 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:23:10.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:10 smithi062 ceph-mon[29666]: pgmap v40: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:23:10.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:10 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:10.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:10 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:10.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:10 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:10.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:10 smithi062 ceph-mon[29666]: from='client.15567 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:23:10.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:10 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/460774774' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:23:10.977 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:23:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:23:10] "GET /metrics HTTP/1.1" 200 38313 "" "Prometheus/2.33.4" 2024-03-24T11:23:11.686 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:11 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:11.687 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:11 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:11.687 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:11 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:11.687 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:11 smithi139 ceph-mon[25719]: pgmap v41: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:23:11.687 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:11 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:11.687 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:11 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T11:23:11.687 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:11 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T11:23:11.687 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:11 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:11.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:11 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:11.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:11 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:11.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:11 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:11.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:11 smithi062 ceph-mon[24799]: pgmap v41: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:23:11.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:11 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:11.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:11 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T11:23:11.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:11 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T11:23:11.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:11 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:11.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:11 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:11.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:11 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:11.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:11 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:11.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:11 smithi062 ceph-mon[29666]: pgmap v41: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:23:11.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:11 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:11.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:11 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T11:23:11.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:11 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T11:23:11.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:11 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:12.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:12 smithi062 ceph-mon[24799]: Upgrade: Updating mgr.x 2024-03-24T11:23:12.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:12 smithi062 ceph-mon[24799]: Deploying daemon mgr.x on smithi139 2024-03-24T11:23:12.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:12 smithi062 ceph-mon[29666]: Upgrade: Updating mgr.x 2024-03-24T11:23:12.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:12 smithi062 ceph-mon[29666]: Deploying daemon mgr.x on smithi139 2024-03-24T11:23:13.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:12 smithi139 ceph-mon[25719]: Upgrade: Updating mgr.x 2024-03-24T11:23:13.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:12 smithi139 ceph-mon[25719]: Deploying daemon mgr.x on smithi139 2024-03-24T11:23:13.904 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:13 smithi139 systemd[1]: Stopping Ceph mgr.x for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:23:14.183 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:13 smithi139 podman[73425]: 2024-03-24 11:23:13.904463542 +0000 UTC m=+0.138182261 container died 7a48bdfa3fc5870ef4fd9567866c2e4672b22b5ba5f31992a6fb8a3cb3b2c0ee (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-db0330b, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-24T11:23:14.183 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:13 smithi139 ceph-mon[25719]: pgmap v42: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:23:14.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:13 smithi062 ceph-mon[24799]: pgmap v42: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:23:14.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:13 smithi062 ceph-mon[29666]: pgmap v42: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:23:14.608 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:14 smithi139 podman[73425]: 2024-03-24 11:23:14.256935183 +0000 UTC m=+0.490653903 container cleanup 7a48bdfa3fc5870ef4fd9567866c2e4672b22b5ba5f31992a6fb8a3cb3b2c0ee (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, RELEASE=squid-db0330b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-24T11:23:14.608 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:14 smithi139 bash[73425]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x 2024-03-24T11:23:14.926 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:14 smithi139 podman[73439]: 2024-03-24 11:23:14.64148204 +0000 UTC m=+0.736383650 container remove 7a48bdfa3fc5870ef4fd9567866c2e4672b22b5ba5f31992a6fb8a3cb3b2c0ee (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x, RELEASE=squid-db0330b, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-24T11:23:14.926 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:14 smithi139 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mgr.x.service: Main process exited, code=exited, status=143/n/a 2024-03-24T11:23:14.926 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:14 smithi139 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mgr.x.service: Failed with result 'exit-code'. 2024-03-24T11:23:14.927 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:14 smithi139 systemd[1]: Stopped Ceph mgr.x for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:23:14.927 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:14 smithi139 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mgr.x.service: Consumed 8.864s CPU time. 2024-03-24T11:23:15.244 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:15 smithi139 systemd[1]: Starting Ceph mgr.x for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:23:15.494 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:15 smithi139 podman[73571]: 2024-03-24 11:23:15.147103048 +0000 UTC m=+0.017706777 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:23:15.495 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:15 smithi139 podman[73571]: 2024-03-24 11:23:15.25916751 +0000 UTC m=+0.129771237 container create d94987b11387352d4adcf75c06f5c3a66cc0dce69cbe5db9b20f3f0acf862650 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-24T11:23:15.858 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:15 smithi139 podman[73571]: 2024-03-24 11:23:15.495200054 +0000 UTC m=+0.365803791 container init d94987b11387352d4adcf75c06f5c3a66cc0dce69cbe5db9b20f3f0acf862650 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5) 2024-03-24T11:23:15.858 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:15 smithi139 podman[73571]: 2024-03-24 11:23:15.498261272 +0000 UTC m=+0.368865009 container start d94987b11387352d4adcf75c06f5c3a66cc0dce69cbe5db9b20f3f0acf862650 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-db0330b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-24T11:23:15.858 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:15 smithi139 ceph-mgr[73590]: -- 172.21.15.139:0/662540998 <== mon.2 v2:172.21.15.139:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c96fc641a0 con 0x55c96fc3a800 2024-03-24T11:23:15.858 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:15 smithi139 bash[73571]: d94987b11387352d4adcf75c06f5c3a66cc0dce69cbe5db9b20f3f0acf862650 2024-03-24T11:23:15.858 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:15 smithi139 systemd[1]: Started Ceph mgr.x for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:23:16.180 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:16 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:16.076+0000 7f6aec086200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-24T11:23:16.180 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:16 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:16.179+0000 7f6aec086200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-24T11:23:16.180 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:15 smithi139 ceph-mon[25719]: pgmap v43: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:23:16.180 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:15 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:16.180 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:15 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:16.180 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:15 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:16.181 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:15 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:16.181 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:15 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:16.181 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:15 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:16.181 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:15 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:16.181 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:15 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]: dispatch 2024-03-24T11:23:16.181 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:15 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]': finished 2024-03-24T11:23:16.181 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:15 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:16.181 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:15 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:16.181 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:15 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:16.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:15 smithi062 ceph-mon[24799]: pgmap v43: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:23:16.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:15 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:16.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:15 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:16.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:15 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:16.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:15 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:16.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:15 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:16.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:15 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:16.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:15 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:16.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:15 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]: dispatch 2024-03-24T11:23:16.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:15 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]': finished 2024-03-24T11:23:16.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:15 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:16.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:15 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:16.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:15 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:16.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:15 smithi062 ceph-mon[29666]: pgmap v43: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:23:16.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:15 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:16.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:15 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:16.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:15 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:16.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:15 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:16.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:15 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:16.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:15 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:16.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:15 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:16.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:15 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]: dispatch 2024-03-24T11:23:16.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:15 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]': finished 2024-03-24T11:23:16.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:15 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:16.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:15 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:16.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:15 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:17.195 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:16 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:16.935+0000 7f6aec086200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-24T11:23:17.195 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:17.043+0000 7f6aec086200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-24T11:23:17.196 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:16 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:17.196 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:16 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:17.196 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:16 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:17.196 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:16 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:17.197 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:16 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:17.197 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:16 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:17.197 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:16 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:17.197 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:16 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:17.197 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:16 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:17.197 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:16 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:17.198 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:16 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:17.198 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:16 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:17.198 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:16 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:17.198 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:16 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:17.198 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:16 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:17.198 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:16 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:17.199 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:16 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:17.199 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:16 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:17.199 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:16 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:17.199 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:16 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:17.199 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:16 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:17.200 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:16 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:17.200 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:16 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:17.200 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:16 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:17.200 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:16 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:17.200 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:16 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:17.201 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:16 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:17.201 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:16 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:17.201 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:16 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:17.202 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:16 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:17.202 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:16 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:17.202 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:16 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:17.202 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:16 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:17.202 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:16 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:17.202 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:16 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:17.203 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:16 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:17.203 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:16 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:17.203 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:16 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:17.203 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:16 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:17.203 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:16 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:17.204 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:16 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:17.204 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:16 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:17.204 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:16 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:17.204 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:16 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:17.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:17.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:17.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:17.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:17.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:17.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:17.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:17.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:17.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:17.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:17.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:17.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:17.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:17.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:17.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:17.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:17.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:17.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:17.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:17.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:17.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:17.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:17.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:17.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:17.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:17.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:17.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:17.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:17.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:17.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:17.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:17.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:17.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:17.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:17.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:17.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:17.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:17.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:17.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:17.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:17.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:17.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:17.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:17.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:17.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:17.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:17.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:17.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:17.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:17.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:17.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:17.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:17.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:17.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:17.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:17.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:17.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:17.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:17.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:17.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:17.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:17.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:17.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:17.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:17.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:17.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:17.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:17.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:17.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:17.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:17.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:17.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:17.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:17.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:17.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:17.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:17.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:17.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:17.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:17.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:17.239 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:17.239 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:17.239 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:17.239 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:17.239 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:17.239 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:17.240 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:17.240 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:16 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:17.481 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:17.194+0000 7f6aec086200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-24T11:23:17.481 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:17.304+0000 7f6aec086200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-24T11:23:17.481 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:17.394+0000 7f6aec086200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-24T11:23:17.799 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:17.680+0000 7f6aec086200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-24T11:23:17.799 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:17.798+0000 7f6aec086200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-24T11:23:18.107 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:17.890+0000 7f6aec086200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-24T11:23:18.108 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:17.995+0000 7f6aec086200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-24T11:23:18.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:18.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:18.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:18.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:18.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:18.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:18.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: pgmap v44: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:23:18.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:18.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:18.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:18.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:18.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:18.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:18.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:18.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:18.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:18.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:18.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:18.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:18.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:23:18.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:18.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:18.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:18.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:18.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:18.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:18.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:23:18.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:18.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:18.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:18.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:18.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:18.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:18.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:18.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:18.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:18.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:18.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:18.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:18.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:18.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:18.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:18.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:18.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:18.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:18.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:18.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:18.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:18.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:18.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:18.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:18.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:18.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:18.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:18.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:18.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:18.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:18.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:18.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:18.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:18.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:18.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:18.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:18.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:18.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:18.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:17 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:18.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:18.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:18.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:18.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:18.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:18.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:18.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: pgmap v44: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:23:18.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:18.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:18.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:18.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:18.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:18.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:18.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:18.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:18.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:18.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:18.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:18.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:18.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:23:18.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:18.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:18.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:18.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:18.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:18.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:18.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:23:18.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:18.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:18.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:18.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:18.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:18.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:18.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:18.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:18.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:18.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: pgmap v44: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:23:18.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:18.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:18.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:18.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:18.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:18.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:18.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:18.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:18.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:18.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:18.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:18.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:18.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:23:18.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:18.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:18.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:18.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:18.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:18.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:18.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:23:18.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:18.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:18.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:18.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:18.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:18.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:18.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:18.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:18.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:18.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:18.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:18.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:18.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:18.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:18.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:18.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:18.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:18.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:18.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:18.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:18.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:18.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:18.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:18.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:18.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:18.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:18.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:18.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:18.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:18.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:18.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:18.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:18.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:18.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:18.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:18.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:18.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:18.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:18.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:18.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:18.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:18.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:18.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:18.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:18.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:18.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:18.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:18.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:18.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:18.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:18.239 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:18.239 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:18.239 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:18.239 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:18.239 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:18.239 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:18.239 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:18.240 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:18.240 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:18.240 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:18.240 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:18.240 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:18.241 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:18.241 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:18.241 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:18.241 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:18.241 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:18.241 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:18.241 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:18.242 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:18.242 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:18.242 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:18.242 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:18.242 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:18.242 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:17 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:18.576 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:18.192+0000 7f6aec086200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-24T11:23:18.857 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:23:18.659Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"smithi062:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"172.21.15.139:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-24T11:23:19.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:19.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:19.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:19.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:19.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:19.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:19.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:19.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:19.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:19.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:19.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:19.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:19.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:19.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:19.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:19.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:19.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:19.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:19.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:19.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:19.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:19.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:19.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:19.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:19.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:19.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:19.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:19.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:19.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:19.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:19.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:19.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:19.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:19.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:19.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:19.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:19.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:19.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:19.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:19.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:19.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:19.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:19.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:19.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:19.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:19.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:19.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:19.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:19.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:19.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:19.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:19.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:19.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:19.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:19.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:19.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:19.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:19.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:19.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:19.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:19.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:19.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:19.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:19.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:19.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:19.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:19.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:19.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:19.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:19.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:19.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:19.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:19.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:19.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:19.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:19.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:19.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:19.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:19.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:19.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:19.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:19.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:19.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:19.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:19.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:19.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:19.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:19.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:19.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:19.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:19.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:19.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:19.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:19.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:19.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:19.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:19.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:19.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:19.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:19.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:19.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:19.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:19.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:19.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:19.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:19.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:19.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:19.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:19.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:19.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:19.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:19.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:19.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:19.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:19.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:19.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:19.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:19.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:19.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:19.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:19.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:19.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:19.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:19.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:19.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:19.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:19.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:19.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:19.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:19.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:19.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:19.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:18 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:19.263 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:19.264 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:19.264 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:19.264 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:19.264 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:19.264 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:19.264 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:19.264 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:19.265 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:19.265 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:19.265 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:19.265 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:19.265 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:19.265 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:19.265 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:19.265 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:19.265 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:19.265 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:19.265 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:19.265 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:19.265 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:19.265 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:19.266 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:19.266 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:19.266 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:19.266 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:19.266 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:19.266 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:19.266 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:19.266 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:19.266 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:19.267 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:19.267 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:19.267 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:19.267 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:19.267 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:19.267 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:19.267 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:19.267 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:19.267 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:19.267 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:19.267 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:19.268 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:19.268 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:19.268 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:19.268 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:19.268 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:19.268 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:19.268 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:19.268 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:19.268 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:19.268 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:19.268 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:19.269 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:19.269 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:19.269 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:19.269 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:19.269 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:19.269 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:19.269 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:19.269 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:19.269 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:19.269 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:19.269 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:19.269 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:19.270 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:18 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:19.576 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:19.262+0000 7f6aec086200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-24T11:23:19.576 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:19.400+0000 7f6aec086200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-24T11:23:19.853 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:19.751+0000 7f6aec086200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-24T11:23:19.853 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:19.852+0000 7f6aec086200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-24T11:23:20.108 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:20.063+0000 7f6aec086200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-24T11:23:20.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:20.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:20.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:20.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:20.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:20.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:20.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:20.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:20.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: pgmap v45: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:23:20.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:20.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:20.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:20.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:20.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:20.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:20.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:20.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:20.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:20.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:20.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:20.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:20.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:20.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:20.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:20.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:20.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:20.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:20.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:20.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:20.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:20.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:20.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:20.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:20.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:20.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:20.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:20.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:20.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:20.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:20.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:20.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:20.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:20.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:20.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:20.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:20.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:20.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:20.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:20.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:20.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:20.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:20.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:20.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:20.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:20.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:20.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:20.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:20.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:20.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:20.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:20.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:20.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:20.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:20.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:20.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:20.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:20.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:19 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:20.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:20.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:20.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:20.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:20.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:20.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:20.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:20.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:20.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: pgmap v45: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:23:20.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:20.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:20.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:20.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:20.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:20.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:20.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:20.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:20.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:20.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:20.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:20.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:20.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:20.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:20.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:20.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:20.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:20.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:20.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:20.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:20.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:20.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:20.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:20.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:20.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:20.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:20.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:20.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:20.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:20.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: pgmap v45: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:23:20.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:20.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:20.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:20.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:20.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:20.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:20.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:20.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:20.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:20.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:20.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:20.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:20.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:20.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:20.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:20.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:20.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:20.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:20.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:20.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:20.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:20.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:20.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:20.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:20.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:20.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:20.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:20.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:20.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:20.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:20.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:20.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:20.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:20.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:20.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:20.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:20.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:20.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:20.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:20.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:20.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:20.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:20.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:20.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:20.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:20.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:20.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:20.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:20.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:20.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:20.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:20.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:20.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:20.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:20.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:20.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:20.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:20.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:20.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:20.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:20.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:20.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:20.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:20.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:20.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:20.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:20.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:20.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:20.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:20.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:20.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:20.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:20.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:20.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:20.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:20.238 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:20.238 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:20.238 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:20.238 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:20.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:20.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:20.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:20.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:20.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:20.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:20.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:20.239 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:20.239 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:20.239 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:20.239 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:20.239 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:20.239 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:20.239 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:20.239 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:20.239 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:19 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:20.407 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:20.264+0000 7f6aec086200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-24T11:23:20.689 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:20.582+0000 7f6aec086200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-24T11:23:20.967 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:20.859+0000 7f6aec086200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-24T11:23:20.967 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:20.968 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:20.968 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:20.968 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:20.968 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:20.968 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:20.968 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:20.968 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:21.226 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:23:20] "GET /metrics HTTP/1.1" 200 38314 "" "Prometheus/2.33.4" 2024-03-24T11:23:21.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:21.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:21.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:21.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:21.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:21.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:21.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:21.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:21.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:21.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:21.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:21.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:21.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:21.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:21.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:21.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:21.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:21.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:21.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:21.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:21.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:21.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:21.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:21.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:21.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:21.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:21.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:21.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:21.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:21.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:21.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:21.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:21.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:21.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:21.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:21.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:21.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:21.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:21.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:21.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:21.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:21.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:21.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:21.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:21.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:21.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:21.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:21.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:21.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:21.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:21.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:21.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:21.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:21.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:21.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:21.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:21.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:21.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:21.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:21.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:21.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:21.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:21.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:21.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:21.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:21.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:21.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:21.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:21.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:21.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:21.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:21.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:21.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:21.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:21.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:21.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:21.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:21.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:21.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:21.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:21.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:21.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:21.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:21.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:21.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:21.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:21.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:21.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:21.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:21.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:21.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:21.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:21.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:21.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:21.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:21.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:21.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:21.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:21.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:21.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:21.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:21.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:21.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:21.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:21.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:21.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:21.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:21.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:21.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:21.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:21.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:21.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:21.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:21.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:21.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:21.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:21.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:21.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:21.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:21.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:21.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:21.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:21.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:21.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:21.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:21.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:21.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:21.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:21.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:21.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:21.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:21.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:20 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:21.357 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:21.121+0000 7f6aec086200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-24T11:23:21.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:21.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:21.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:21.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:21.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:21.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:21.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:21.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:21.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:21.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:21.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:21.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:21.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:21.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:21.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:21.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:21.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:21.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:21.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:21.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:21.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:21.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:21.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:21.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:21.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:21.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:21.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:21.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:21.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:21.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:21.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:21.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:21.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:21.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:21.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:21.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:21.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:21.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:21.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:21.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:21.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:21.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:21.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:21.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:21.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:21.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:21.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:21.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:21.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:21.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:21.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:21.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:21.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:21.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:21.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:21.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:21.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:21.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:20 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:22.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:22.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:22.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:22.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:22.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:22.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:22.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:22.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:22.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: pgmap v46: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-24T11:23:22.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:22.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:22.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:22.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:22.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:22.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:22.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:22.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:22.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:22.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:22.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:22.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:22.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:22.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:22.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:22.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:22.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:22.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:22.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:22.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:22.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:22.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:22.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:22.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:22.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:22.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:22.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:22.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:22.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:22.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:22.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:22.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:22.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:22.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:22.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:22.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:22.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:22.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:22.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:22.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:22.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:22.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:22.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:22.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:22.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:22.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:22.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:22.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:22.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:22.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:22.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:22.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:22.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:22.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:22.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:22.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:22.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:22.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:22.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:22.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:22.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:22.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: pgmap v46: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-24T11:23:22.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:22.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:22.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:22.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:22.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:22.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:22.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:22.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:22.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:22.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:22.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:22.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:22.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:22.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:22.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:22.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:22.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:22.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:22.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:22.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:22.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:22.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:22.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:22.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:22.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:22.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:22.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:22.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:22.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:22.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:22.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:22.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:22.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:22.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:22.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:22.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:22.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:22.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:22.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:22.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:22.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:22.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:22.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:22.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:22.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:22.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:22.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:22.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:22.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:22.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:22.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:22.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:22.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:21 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:22.295 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:22.031+0000 7f6aec086200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-24T11:23:22.295 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:22.295 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:22.296 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:22.296 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:22.296 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:22.296 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:22.296 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:22.296 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:22.296 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: pgmap v46: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-24T11:23:22.296 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:22.296 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:22.296 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:22.296 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:22.296 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:22.296 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:22.297 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:22.297 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:22.297 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:22.297 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:22.297 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:22.297 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:22.297 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:22.297 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:22.297 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:22.297 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:22.297 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:22.297 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:22.297 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:22.297 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:22.298 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:22.298 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:22.298 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:22.298 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:22.298 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:22.298 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:22.298 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:22.298 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:22.299 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:22.299 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:22.299 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:22.299 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:22.299 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:22.299 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:22.299 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:22.299 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:22.299 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:22.299 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:22.299 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:22.299 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:22.299 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:22.299 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:22.300 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:22.300 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:22.300 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:22.300 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:22.300 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:22.300 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:22.300 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:22.300 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:22.300 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:22.300 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:22.300 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:21 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:22.922 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:22.649+0000 7f6aec086200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-24T11:23:22.922 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:22.759+0000 7f6aec086200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-24T11:23:23.256 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:22.998+0000 7f6aec086200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-24T11:23:23.256 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: [24/Mar/2024:11:23:23] ENGINE Bus STARTING 2024-03-24T11:23:23.256 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: CherryPy Checker: 2024-03-24T11:23:23.256 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: The Application mounted at '' has an empty config. 2024-03-24T11:23:23.256 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:23.256 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: [24/Mar/2024:11:23:23] ENGINE Serving on http://:::9283 2024-03-24T11:23:23.257 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: [24/Mar/2024:11:23:23] ENGINE Bus STARTED 2024-03-24T11:23:23.257 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:23.257 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:23.257 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:23.257 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:23.257 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:23.257 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:23.257 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:23.258 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:23.258 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:23.258 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:23.258 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:23.258 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:23.258 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:23.258 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:23.258 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:23.258 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:23.258 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:23.258 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:23.258 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:23.258 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:23.258 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:23.259 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:23.259 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:23.259 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:23.259 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:23.259 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:23.259 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:23.259 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:23.259 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:23.259 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:23.259 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:23.259 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:23.259 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:23.259 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:23.259 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:23.260 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:23.260 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:23.260 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:23.260 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:23.260 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:23.260 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:23.260 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:23.260 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:23.260 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:23.260 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:23.260 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:23.260 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:23.260 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:23.260 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:23.261 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:23.261 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:23.261 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:23.261 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:23.261 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:23.261 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:23.261 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:23.261 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:23.261 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:23.261 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:23.261 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:22 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:23.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:23.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:23.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:23.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:23.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:23.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:23.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:23.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:23.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:23.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:23.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:23.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:23.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:23.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:23.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:23.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:23.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:23.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:23.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:23.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:23.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:23.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:23.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:23.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:23.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:23.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:23.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:23.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:23.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:23.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:23.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:23.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:23.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:23.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:23.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:23.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:23.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:23.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:23.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:23.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:23.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:23.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:23.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:23.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:23.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:23.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:23.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:23.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:23.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:23.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:23.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:23.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:23.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:23.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:23.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:23.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:23.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:23.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:23.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:23.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:23.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:23.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:23.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:23.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:23.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:23.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:23.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:23.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:23.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:23.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:23.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:23.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:23.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:23.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:23.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:23.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:23.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:23.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:23.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:23.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:23.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:23.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:23.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:23.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:23.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:23.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:23.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:23.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:23.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:23.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:23.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:23.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:23.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:23.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:23.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:23.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:23.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:23.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:23.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:23.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:23.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:23.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:23.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:23.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:23.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:23.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:23.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:23.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:23.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:23.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:23.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:23.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:23.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:23.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:23.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:23.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:23.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:23.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:23.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:23.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:22 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:24.181 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:24.181 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:24.181 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:24.181 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:24.182 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:24.182 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:24.182 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:24.182 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:24.182 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: pgmap v47: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-24T11:23:24.182 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:24.182 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:24.182 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:24.182 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:24.182 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:24.182 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:24.182 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: Standby manager daemon x restarted 2024-03-24T11:23:24.182 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: Standby manager daemon x started 2024-03-24T11:23:24.182 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.? 172.21.15.139:0/2386341945' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-24T11:23:24.182 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.? 172.21.15.139:0/2386341945' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T11:23:24.183 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.? 172.21.15.139:0/2386341945' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-24T11:23:24.183 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.? 172.21.15.139:0/2386341945' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T11:23:24.183 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:24.183 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:24.183 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:24.183 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:24.183 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:24.183 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:24.183 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:24.183 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:24.183 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:24.183 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:24.183 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:24.183 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:24.183 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:24.183 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:24.184 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:24.184 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:24.184 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:24.184 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:24.184 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:24.184 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:24.184 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:24.184 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:24.184 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:24.184 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:24.184 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:24.184 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:24.184 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:24.184 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:24.184 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:24.185 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:24.185 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:24.185 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:24.185 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:24.185 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:24.185 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:24.185 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:24.185 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:24.185 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:24.185 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:24.185 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:24.185 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:24.185 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:24.185 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:24.186 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:24.186 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:24.186 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:24.186 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:24.186 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:24.186 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:24.186 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:24.186 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:24.186 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:24.186 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:24.186 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:24.186 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:24.186 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:24.186 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:23 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:24.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:24.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:24.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:24.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:24.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:24.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:24.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:24.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:24.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: pgmap v47: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-24T11:23:24.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:24.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:24.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:24.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:24.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:24.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:24.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: Standby manager daemon x restarted 2024-03-24T11:23:24.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: Standby manager daemon x started 2024-03-24T11:23:24.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.? 172.21.15.139:0/2386341945' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-24T11:23:24.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.? 172.21.15.139:0/2386341945' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T11:23:24.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.? 172.21.15.139:0/2386341945' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-24T11:23:24.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.? 172.21.15.139:0/2386341945' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T11:23:24.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:24.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:24.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:24.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:24.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:24.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:24.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:24.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:24.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:24.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:24.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:24.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:24.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:24.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:24.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:24.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:24.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:24.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:24.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:24.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:24.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:24.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:24.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:24.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:24.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:24.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:24.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:24.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:24.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:24.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:24.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:24.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:24.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:24.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:24.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:24.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:24.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:24.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:24.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:24.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:24.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:24.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:24.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:24.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:24.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:24.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:24.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: pgmap v47: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-24T11:23:24.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:24.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:24.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:24.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:24.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:24.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:24.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: Standby manager daemon x restarted 2024-03-24T11:23:24.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: Standby manager daemon x started 2024-03-24T11:23:24.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.? 172.21.15.139:0/2386341945' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-24T11:23:24.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.? 172.21.15.139:0/2386341945' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T11:23:24.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.? 172.21.15.139:0/2386341945' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-24T11:23:24.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.? 172.21.15.139:0/2386341945' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T11:23:24.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:24.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:24.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:24.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:24.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:24.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:24.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:24.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:24.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:24.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:24.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:24.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:24.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:24.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:24.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:24.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:24.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:24.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:24.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:24.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:24.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:24.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:24.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:24.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:24.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:24.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:24.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:24.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:24.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:24.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:24.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:24.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:24.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:24.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:24.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:24.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:24.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:24.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:24.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:24.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:24.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:24.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:24.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:24.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:24.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:24.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:24.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:24.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:24.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:24.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:24.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:24.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:24.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:24.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:24.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:24.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:24.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:24.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:24.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:24.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:24.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:24.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:24.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:24.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:24.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:24.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:24.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:24.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:24.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:24.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:24.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:24.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:24.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:24.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:24.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:24.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:23 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:25.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:25.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:25.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:25.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:25.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:25.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:25.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:25.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:25.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:25.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: mgrmap e37: y(active, since 82s), standbys: x 2024-03-24T11:23:25.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:25.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:25.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:25.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:25.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:25.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:25.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:25.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:25.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:25.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:25.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:25.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:25.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:25.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:25.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:25.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:25.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:25.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:25.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:25.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:25.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:25.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:25.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:25.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:25.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:25.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:25.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:25.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:25.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:25.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:25.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:25.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:25.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:25.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:25.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:25.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:25.365 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:25.365 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:25.365 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:25.365 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:25.365 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:25.365 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:25.365 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:25.365 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:25.365 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:25.365 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:25.365 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:25.365 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:25.365 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:25.365 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:25.366 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:25.366 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:25.366 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:25.366 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:25.366 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:25.366 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:25.366 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:25.366 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:25.366 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:25.366 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:25.366 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:25.366 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:25.366 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:25 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:25.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:25.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:25.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:25.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:25.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:25.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:25.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:25.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:25.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:25.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: mgrmap e37: y(active, since 82s), standbys: x 2024-03-24T11:23:25.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:25.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:25.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:25.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:25.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:25.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:25.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:25.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:25.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:25.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:25.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:25.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:25.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:25.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:25.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:25.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:25.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:25.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:25.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:25.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:25.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:25.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:25.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:25.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:25.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:25.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:25.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:25.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:25.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:25.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:25.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: mgrmap e37: y(active, since 82s), standbys: x 2024-03-24T11:23:25.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:25.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:25.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:25.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:25.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:25.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:25.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:25.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:25.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:25.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:25.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:25.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:25.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:25.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:25.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:25.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:25.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:25.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:25.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:25.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:25.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:25.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:25.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:25.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:25.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:25.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:25.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:25.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:25.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:25.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:25.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:25.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:25.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:25.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:25.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:25.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:25.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:25.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:25.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:25.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:25.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:25.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:25.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:25.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:25.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:25.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:25.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:25.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:25.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:25.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:25.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:25.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:25.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:25.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:25.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:25.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:25.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:25.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:25.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:25.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:25.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:25.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:25.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:25.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:25.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:25.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:25.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:25.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:25.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:25.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:25.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:25.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:25.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:25.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:25.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:25.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:25.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:25.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:25.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:25.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:25.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:25.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:25.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:25.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:25.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:25.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:25.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:25.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:25.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:25.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:25.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:25.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:25.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:25.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:25.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:25.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:25.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:25.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:25.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:25.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:25.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:25.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:25.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:25.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:25.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:25 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:26.014 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:26.014 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:26.014 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:26.014 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:26.014 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:26.014 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:26.014 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:26.015 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:26.015 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: pgmap v48: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:23:26.015 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:26.015 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:26.015 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:26.015 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:26.015 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:26.015 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:26.015 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:26.015 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:26.015 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:26.015 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:26.015 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:26.015 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:26.015 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:26.016 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:26.016 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:26.016 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:26.016 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:26.016 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:26.016 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:26.016 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:26.016 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:26.016 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:26.016 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:26.017 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:26.017 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:26.017 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:26.017 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:26.017 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:26.017 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:26.017 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:26.017 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: pgmap v48: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:23:26.017 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:26.017 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:26.017 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:26.017 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:26.017 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:26.017 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:26.018 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:26.018 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:26.018 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:26.018 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:26.018 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:26.018 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:26.018 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:26.018 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:26.018 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:26.018 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:26.018 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:26.018 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:26.018 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:26.018 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:26.018 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:26.019 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:26.019 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:26.019 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:26.019 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:26.019 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:26.019 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:26.019 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:26.019 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:26.020 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:26.020 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:26.020 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:26.020 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:26.020 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:26.020 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:26.020 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:26.020 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:26.020 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:26.020 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:26.020 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:26.020 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:26.020 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:26.020 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:26.020 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:26.021 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:26.021 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:26.021 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:26.021 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:26.021 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:26.021 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:26.021 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:26.021 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:26.021 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:26.021 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:26.021 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:26.021 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:26.021 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:26.021 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:26.022 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:26.022 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:26.022 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:26.022 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:26.022 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:26.022 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:26.022 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:26.022 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:26.022 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:26.023 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:26.023 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:26.023 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:26.023 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:26.023 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:26.023 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:26.023 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:26.023 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:26.023 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:26.023 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:26.023 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:26.023 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:26.023 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:26.023 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:26.024 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:26.024 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:26.024 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:26.024 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:26.024 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:26.024 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:26.024 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:26.024 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:26.024 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:26.024 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:26.024 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:26.024 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:26.024 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:26.024 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:26.024 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:26 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:26.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:26.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:26.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:26.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:26.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:26.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:26.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:26.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:26.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: pgmap v48: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:23:26.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:26.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:26.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:26.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:26.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:26.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:26.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:26.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:26.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:26.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:26.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:26.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:26.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:26.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:26.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:26.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:26.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:26.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:26.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:26.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:26.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:26.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:26.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:26.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:26.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:26.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:26.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:26.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:26.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:26.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:26.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:26.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:26.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:26.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:26.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:26.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:26.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:26.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:26.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:26.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:26.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:26.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:26.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:26.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:26.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:26.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:26.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:26.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:26.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:26.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:26.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:26.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:26.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:26.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:26.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:26.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:26.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:26.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:26.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:26 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:27.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:27 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:27.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:27 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:27.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:27 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:27.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:27 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:27.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:27 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:27.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:27 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:27.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:27 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:27.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:27 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:27.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:27 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:27.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:27 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:27.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:27 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:27.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:27 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:27.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:27 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:27.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:27 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-24T11:23:27.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:27 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "mon ok-to-stop", "ids": ["a"]}]: dispatch 2024-03-24T11:23:27.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:27 smithi139 ceph-mon[25719]: Upgrade: It appears safe to stop mon.a 2024-03-24T11:23:27.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:27 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:27.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:27 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:27.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:27 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:27.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:27 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:27.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:27 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:27.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:27 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:27.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:27 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:27.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:27 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:27.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:27 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:27.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:27 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:27.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:27 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:27.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:27 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:27.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:27 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:27.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:27 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:27.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:27 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:27.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:27 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:27.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:27 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-24T11:23:27.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:27 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "mon ok-to-stop", "ids": ["a"]}]: dispatch 2024-03-24T11:23:27.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:27 smithi062 ceph-mon[29666]: Upgrade: It appears safe to stop mon.a 2024-03-24T11:23:27.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:27 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:27.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:27 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:27.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:27 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:27.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:27 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:27.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:27 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:27.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:27 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:27.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:27 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:27.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:27 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:27.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:27 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:27.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:27 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:27.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:27 smithi062 ceph-mon[24799]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:27.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:27 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:27.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:27 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:27.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:27 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:27.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:27 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:27.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:27 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:27.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:27 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-24T11:23:27.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:27 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "mon ok-to-stop", "ids": ["a"]}]: dispatch 2024-03-24T11:23:27.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:27 smithi062 ceph-mon[24799]: Upgrade: It appears safe to stop mon.a 2024-03-24T11:23:27.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:27 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:27.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:27 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:27.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:27 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:28.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:28 smithi139 ceph-mon[25719]: pgmap v49: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-24T11:23:28.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:28 smithi139 ceph-mon[25719]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:23:28.399 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:28 smithi062 ceph-mon[24799]: pgmap v49: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-24T11:23:28.399 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:28 smithi062 ceph-mon[24799]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:23:28.400 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:28 smithi062 ceph-mon[29666]: pgmap v49: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-24T11:23:28.400 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:28 smithi062 ceph-mon[29666]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:23:28.968 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:23:28 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:23:28.659Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"smithi062:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"172.21.15.139:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-24T11:23:29.445 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:29 smithi062 ceph-mon[29666]: Upgrade: Updating mon.a 2024-03-24T11:23:29.445 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:29.445 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T11:23:29.445 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T11:23:29.445 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:29 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:29.445 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:29 smithi062 ceph-mon[29666]: Deploying daemon mon.a on smithi062 2024-03-24T11:23:29.445 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:29 smithi062 ceph-mon[29666]: pgmap v50: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:23:29.446 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:29 smithi062 ceph-mon[24799]: Upgrade: Updating mon.a 2024-03-24T11:23:29.446 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:29.446 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T11:23:29.446 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T11:23:29.446 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:29 smithi062 ceph-mon[24799]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:29.446 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:29 smithi062 ceph-mon[24799]: Deploying daemon mon.a on smithi062 2024-03-24T11:23:29.446 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:29 smithi062 ceph-mon[24799]: pgmap v50: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:23:29.553 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:29 smithi139 ceph-mon[25719]: Upgrade: Updating mon.a 2024-03-24T11:23:29.553 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' 2024-03-24T11:23:29.554 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T11:23:29.554 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T11:23:29.554 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:29 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:29.554 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:29 smithi139 ceph-mon[25719]: Deploying daemon mon.a on smithi062 2024-03-24T11:23:29.554 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:29 smithi139 ceph-mon[25719]: pgmap v50: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:23:30.881 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:30 smithi062 systemd[1]: Stopping Ceph mon.a for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:23:30.882 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:23:30 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:23:30] "GET /metrics HTTP/1.1" 200 38314 "" "Prometheus/2.33.4" 2024-03-24T11:23:31.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:30 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-a[24795]: 2024-03-24T11:23:30.881+0000 7fc55d43c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T11:23:31.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:30 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-a[24795]: 2024-03-24T11:23:30.881+0000 7fc55d43c700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-24T11:23:31.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:30 smithi062 podman[86608]: 2024-03-24 11:23:30.952994005 +0000 UTC m=+0.179911268 container died 712f01aa91804babf25132e8257ba078ccc0905c25c6c191c3daa9e15b989df4 (image=quay.io/ceph/ceph:v17.2.0, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-a, GIT_CLEAN=True, build-date=2022-05-03T08:36:31.336870, io.openshift.expose-services=, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., name=centos-stream, distribution-scope=public, version=8, io.buildah.version=1.19.8, io.openshift.tags=base centos centos-stream, vcs-type=git, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, ceph=True, io.k8s.display-name=CentOS Stream 8, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_BRANCH=HEAD, release=754, com.redhat.component=centos-stream-container, CEPH_POINT_RELEASE=-17.2.0, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, architecture=x86_64, RELEASE=HEAD, maintainer=Guillaume Abrioux , description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, vendor=Red Hat, Inc., vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb) 2024-03-24T11:23:31.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:23:30 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: [24/Mar/2024:11:23:30] ENGINE Bus STOPPING 2024-03-24T11:23:31.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:23:30 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: [24/Mar/2024:11:23:30] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-03-24T11:23:31.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:23:30 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: [24/Mar/2024:11:23:30] ENGINE Bus STOPPED 2024-03-24T11:23:31.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:23:30 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: [24/Mar/2024:11:23:30] ENGINE Bus STARTING 2024-03-24T11:23:31.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:23:31 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: [24/Mar/2024:11:23:31] ENGINE Serving on http://:::9283 2024-03-24T11:23:31.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:23:31 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: [24/Mar/2024:11:23:31] ENGINE Bus STARTED 2024-03-24T11:23:31.526 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:31 smithi062 podman[86608]: 2024-03-24 11:23:31.323088368 +0000 UTC m=+0.550005632 container cleanup 712f01aa91804babf25132e8257ba078ccc0905c25c6c191c3daa9e15b989df4 (image=quay.io/ceph/ceph:v17.2.0, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-a, RELEASE=HEAD, io.openshift.expose-services=, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, architecture=x86_64, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, GIT_BRANCH=HEAD, ceph=True, io.openshift.tags=base centos centos-stream, maintainer=Guillaume Abrioux , vendor=Red Hat, Inc., distribution-scope=public, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, vcs-type=git, build-date=2022-05-03T08:36:31.336870, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., name=centos-stream, CEPH_POINT_RELEASE=-17.2.0, io.buildah.version=1.19.8, release=754, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, io.k8s.display-name=CentOS Stream 8, GIT_REPO=https://github.com/ceph/ceph-container.git, version=8, GIT_CLEAN=True, com.redhat.component=centos-stream-container) 2024-03-24T11:23:31.526 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:31 smithi062 bash[86608]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-a 2024-03-24T11:23:31.847 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:31 smithi062 podman[86633]: 2024-03-24 11:23:31.524217456 +0000 UTC m=+0.569140782 container remove 712f01aa91804babf25132e8257ba078ccc0905c25c6c191c3daa9e15b989df4 (image=quay.io/ceph/ceph:v17.2.0, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-a, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.19.8, release=754, RELEASE=HEAD, CEPH_POINT_RELEASE=-17.2.0, GIT_BRANCH=HEAD, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, io.k8s.display-name=CentOS Stream 8, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, vendor=Red Hat, Inc., version=8, architecture=x86_64, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, vcs-type=git, io.openshift.expose-services=, GIT_CLEAN=True, io.openshift.tags=base centos centos-stream, com.redhat.component=centos-stream-container, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., ceph=True, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, build-date=2022-05-03T08:36:31.336870, name=centos-stream, distribution-scope=public, maintainer=Guillaume Abrioux , vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb) 2024-03-24T11:23:31.848 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:31 smithi062 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-24T11:23:31.848 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:31 smithi062 systemd[1]: Stopped Ceph mon.a for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:23:31.848 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:31 smithi062 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mon.a.service: Consumed 18.901s CPU time. 2024-03-24T11:23:32.167 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:31 smithi062 systemd[1]: Starting Ceph mon.a for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:23:32.445 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 podman[86766]: 2024-03-24 11:23:32.070129997 +0000 UTC m=+0.018735303 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:23:32.445 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 podman[86766]: 2024-03-24 11:23:32.178821343 +0000 UTC m=+0.127426690 container create 37e940608c6af0a2b9b0cbd02a50be0676841f7673f003b5cc6431b002ee1d39 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-a, RELEASE=squid-db0330b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-24T11:23:32.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 podman[86766]: 2024-03-24 11:23:32.445143041 +0000 UTC m=+0.393748356 container init 37e940608c6af0a2b9b0cbd02a50be0676841f7673f003b5cc6431b002ee1d39 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-a, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=squid-db0330b, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS) 2024-03-24T11:23:32.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 podman[86766]: 2024-03-24 11:23:32.448278355 +0000 UTC m=+0.396883661 container start 37e940608c6af0a2b9b0cbd02a50be0676841f7673f003b5cc6431b002ee1d39 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, RELEASE=squid-db0330b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True) 2024-03-24T11:23:32.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: set uid:gid to 167:167 (ceph:ceph) 2024-03-24T11:23:32.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev), process ceph-mon, pid 2 2024-03-24T11:23:32.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: pidfile_write: ignore empty --pid-file 2024-03-24T11:23:32.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: load: jerasure load: lrc 2024-03-24T11:23:32.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: RocksDB version: 7.9.2 2024-03-24T11:23:32.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Git sha 0 2024-03-24T11:23:32.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Compile date 2024-03-16 10:36:11 2024-03-24T11:23:32.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: DB SUMMARY 2024-03-24T11:23:32.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: DB Session ID: 05XMXXZKHVVKY6A77WZ3 2024-03-24T11:23:32.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: CURRENT file: CURRENT 2024-03-24T11:23:32.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: IDENTITY file: IDENTITY 2024-03-24T11:23:32.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: MANIFEST file: MANIFEST-000015 size: 1922 Bytes 2024-03-24T11:23:32.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000045.sst 2024-03-24T11:23:32.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000043.log size: 797899 ; 2024-03-24T11:23:32.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.error_if_exists: 0 2024-03-24T11:23:32.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.create_if_missing: 0 2024-03-24T11:23:32.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.paranoid_checks: 1 2024-03-24T11:23:32.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-24T11:23:32.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-24T11:23:32.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-24T11:23:32.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.env: 0x563cfe9c2ae0 2024-03-24T11:23:32.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.fs: PosixFileSystem 2024-03-24T11:23:32.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.info_log: 0x563d010038c0 2024-03-24T11:23:32.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.max_file_opening_threads: 16 2024-03-24T11:23:32.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.statistics: (nil) 2024-03-24T11:23:32.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.use_fsync: 0 2024-03-24T11:23:32.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.max_log_file_size: 0 2024-03-24T11:23:32.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-24T11:23:32.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-24T11:23:32.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.keep_log_file_num: 1000 2024-03-24T11:23:32.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.recycle_log_file_num: 0 2024-03-24T11:23:32.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.allow_fallocate: 1 2024-03-24T11:23:32.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.allow_mmap_reads: 0 2024-03-24T11:23:32.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.allow_mmap_writes: 0 2024-03-24T11:23:32.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.use_direct_reads: 0 2024-03-24T11:23:32.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-24T11:23:32.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.create_missing_column_families: 0 2024-03-24T11:23:32.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.db_log_dir: 2024-03-24T11:23:32.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.wal_dir: 2024-03-24T11:23:32.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-24T11:23:32.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-24T11:23:32.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-24T11:23:32.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-24T11:23:32.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-24T11:23:32.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-24T11:23:32.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.advise_random_on_open: 1 2024-03-24T11:23:32.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.db_write_buffer_size: 0 2024-03-24T11:23:32.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.write_buffer_manager: 0x563d00fd6dc0 2024-03-24T11:23:32.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-24T11:23:32.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-24T11:23:32.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-24T11:23:32.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.rate_limiter: (nil) 2024-03-24T11:23:32.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-24T11:23:32.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.wal_recovery_mode: 2 2024-03-24T11:23:32.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.enable_thread_tracking: 0 2024-03-24T11:23:32.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.enable_pipelined_write: 0 2024-03-24T11:23:32.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.unordered_write: 0 2024-03-24T11:23:32.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-24T11:23:32.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-24T11:23:32.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-24T11:23:32.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-24T11:23:32.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.row_cache: None 2024-03-24T11:23:32.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.wal_filter: None 2024-03-24T11:23:32.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-24T11:23:32.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.allow_ingest_behind: 0 2024-03-24T11:23:32.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.two_write_queues: 0 2024-03-24T11:23:32.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.manual_wal_flush: 0 2024-03-24T11:23:32.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.wal_compression: 0 2024-03-24T11:23:32.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.atomic_flush: 0 2024-03-24T11:23:32.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-24T11:23:32.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-24T11:23:32.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-24T11:23:32.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.log_readahead_size: 0 2024-03-24T11:23:32.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-24T11:23:32.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.best_efforts_recovery: 0 2024-03-24T11:23:32.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-24T11:23:32.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-24T11:23:32.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.allow_data_in_errors: 0 2024-03-24T11:23:32.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.db_host_id: __hostname__ 2024-03-24T11:23:32.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-24T11:23:32.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.max_background_jobs: 2 2024-03-24T11:23:32.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.max_background_compactions: -1 2024-03-24T11:23:32.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.max_subcompactions: 1 2024-03-24T11:23:32.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-24T11:23:32.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-24T11:23:32.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-24T11:23:32.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.max_total_wal_size: 0 2024-03-24T11:23:32.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-24T11:23:32.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-24T11:23:32.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-24T11:23:32.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-24T11:23:32.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.max_open_files: -1 2024-03-24T11:23:32.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.bytes_per_sync: 0 2024-03-24T11:23:32.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-24T11:23:32.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-24T11:23:32.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.compaction_readahead_size: 0 2024-03-24T11:23:32.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.max_background_flushes: -1 2024-03-24T11:23:32.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Compression algorithms supported: 2024-03-24T11:23:32.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-24T11:23:32.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: kZSTD supported: 0 2024-03-24T11:23:32.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: kXpressCompression supported: 0 2024-03-24T11:23:32.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: kLZ4HCCompression supported: 1 2024-03-24T11:23:32.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: kZlibCompression supported: 1 2024-03-24T11:23:32.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: kSnappyCompression supported: 1 2024-03-24T11:23:32.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: kLZ4Compression supported: 1 2024-03-24T11:23:32.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: kBZip2Compression supported: 0 2024-03-24T11:23:32.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-24T11:23:32.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-24T11:23:32.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000015 2024-03-24T11:23:32.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-24T11:23:32.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-24T11:23:32.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.merge_operator: 2024-03-24T11:23:32.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.compaction_filter: None 2024-03-24T11:23:32.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.compaction_filter_factory: None 2024-03-24T11:23:32.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.sst_partitioner_factory: None 2024-03-24T11:23:32.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-24T11:23:32.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-24T11:23:32.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x563d010039c0) 2024-03-24T11:23:32.737 INFO:journalctl@ceph.mon.a.smithi062.stdout: cache_index_and_filter_blocks: 1 2024-03-24T11:23:32.737 INFO:journalctl@ceph.mon.a.smithi062.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-24T11:23:32.737 INFO:journalctl@ceph.mon.a.smithi062.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-24T11:23:32.737 INFO:journalctl@ceph.mon.a.smithi062.stdout: pin_top_level_index_and_filter: 1 2024-03-24T11:23:32.737 INFO:journalctl@ceph.mon.a.smithi062.stdout: index_type: 0 2024-03-24T11:23:32.737 INFO:journalctl@ceph.mon.a.smithi062.stdout: data_block_index_type: 0 2024-03-24T11:23:32.737 INFO:journalctl@ceph.mon.a.smithi062.stdout: index_shortening: 1 2024-03-24T11:23:32.737 INFO:journalctl@ceph.mon.a.smithi062.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-24T11:23:32.737 INFO:journalctl@ceph.mon.a.smithi062.stdout: checksum: 4 2024-03-24T11:23:32.737 INFO:journalctl@ceph.mon.a.smithi062.stdout: no_block_cache: 0 2024-03-24T11:23:32.737 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_cache: 0x563d01007090 2024-03-24T11:23:32.738 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_cache_name: BinnedLRUCache 2024-03-24T11:23:32.738 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_cache_options: 2024-03-24T11:23:32.738 INFO:journalctl@ceph.mon.a.smithi062.stdout: capacity : 536870912 2024-03-24T11:23:32.738 INFO:journalctl@ceph.mon.a.smithi062.stdout: num_shard_bits : 4 2024-03-24T11:23:32.738 INFO:journalctl@ceph.mon.a.smithi062.stdout: strict_capacity_limit : 0 2024-03-24T11:23:32.738 INFO:journalctl@ceph.mon.a.smithi062.stdout: high_pri_pool_ratio: 0.000 2024-03-24T11:23:32.738 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_cache_compressed: (nil) 2024-03-24T11:23:32.738 INFO:journalctl@ceph.mon.a.smithi062.stdout: persistent_cache: (nil) 2024-03-24T11:23:32.738 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_size: 4096 2024-03-24T11:23:32.738 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_size_deviation: 10 2024-03-24T11:23:32.738 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_restart_interval: 16 2024-03-24T11:23:32.738 INFO:journalctl@ceph.mon.a.smithi062.stdout: index_block_restart_interval: 1 2024-03-24T11:23:32.738 INFO:journalctl@ceph.mon.a.smithi062.stdout: metadata_block_size: 4096 2024-03-24T11:23:32.738 INFO:journalctl@ceph.mon.a.smithi062.stdout: partition_filters: 0 2024-03-24T11:23:32.739 INFO:journalctl@ceph.mon.a.smithi062.stdout: use_delta_encoding: 1 2024-03-24T11:23:32.739 INFO:journalctl@ceph.mon.a.smithi062.stdout: filter_policy: bloomfilter 2024-03-24T11:23:32.739 INFO:journalctl@ceph.mon.a.smithi062.stdout: whole_key_filtering: 1 2024-03-24T11:23:32.739 INFO:journalctl@ceph.mon.a.smithi062.stdout: verify_compression: 0 2024-03-24T11:23:32.739 INFO:journalctl@ceph.mon.a.smithi062.stdout: read_amp_bytes_per_bit: 0 2024-03-24T11:23:32.739 INFO:journalctl@ceph.mon.a.smithi062.stdout: format_version: 5 2024-03-24T11:23:32.739 INFO:journalctl@ceph.mon.a.smithi062.stdout: enable_index_compression: 1 2024-03-24T11:23:32.739 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_align: 0 2024-03-24T11:23:32.739 INFO:journalctl@ceph.mon.a.smithi062.stdout: max_auto_readahead_size: 262144 2024-03-24T11:23:32.739 INFO:journalctl@ceph.mon.a.smithi062.stdout: prepopulate_block_cache: 0 2024-03-24T11:23:32.739 INFO:journalctl@ceph.mon.a.smithi062.stdout: initial_auto_readahead_size: 8192 2024-03-24T11:23:32.739 INFO:journalctl@ceph.mon.a.smithi062.stdout: num_file_reads_for_auto_readahead: 2 2024-03-24T11:23:32.739 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.write_buffer_size: 33554432 2024-03-24T11:23:32.739 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.max_write_buffer_number: 2 2024-03-24T11:23:32.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.compression: NoCompression 2024-03-24T11:23:32.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.bottommost_compression: Disabled 2024-03-24T11:23:32.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.prefix_extractor: nullptr 2024-03-24T11:23:32.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-24T11:23:32.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.num_levels: 7 2024-03-24T11:23:32.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-24T11:23:32.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-24T11:23:32.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-24T11:23:32.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-24T11:23:32.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-24T11:23:32.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-24T11:23:32.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-24T11:23:32.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-24T11:23:32.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-24T11:23:32.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-24T11:23:32.741 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-24T11:23:32.741 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-24T11:23:32.741 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-24T11:23:32.741 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.compression_opts.level: 32767 2024-03-24T11:23:32.741 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.compression_opts.strategy: 0 2024-03-24T11:23:32.741 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-24T11:23:32.741 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-24T11:23:32.741 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-24T11:23:32.741 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-24T11:23:32.741 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.compression_opts.enabled: false 2024-03-24T11:23:32.741 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-24T11:23:32.741 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-24T11:23:32.741 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-24T11:23:32.741 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-24T11:23:32.742 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.target_file_size_base: 67108864 2024-03-24T11:23:32.742 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-24T11:23:32.742 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-24T11:23:32.742 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-24T11:23:32.742 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-24T11:23:32.742 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-24T11:23:32.742 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-24T11:23:32.742 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-24T11:23:32.742 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-24T11:23:32.742 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-24T11:23:32.742 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-24T11:23:32.742 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-24T11:23:32.742 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-24T11:23:32.743 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-24T11:23:32.743 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-24T11:23:32.743 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.arena_block_size: 1048576 2024-03-24T11:23:32.743 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-24T11:23:32.743 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-24T11:23:32.743 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.disable_auto_compactions: 0 2024-03-24T11:23:32.743 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-24T11:23:32.743 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-24T11:23:32.743 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-24T11:23:32.743 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-24T11:23:32.743 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-24T11:23:32.743 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-24T11:23:32.743 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-24T11:23:32.744 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-24T11:23:32.744 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-24T11:23:32.744 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-24T11:23:32.744 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.table_properties_collectors: 2024-03-24T11:23:32.744 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.inplace_update_support: 0 2024-03-24T11:23:32.745 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-24T11:23:32.745 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-24T11:23:32.745 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-24T11:23:32.745 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-24T11:23:32.745 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.bloom_locality: 0 2024-03-24T11:23:32.745 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.max_successive_merges: 0 2024-03-24T11:23:32.745 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-24T11:23:32.745 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.paranoid_file_checks: 0 2024-03-24T11:23:32.745 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.force_consistency_checks: 1 2024-03-24T11:23:32.745 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.report_bg_io_stats: 0 2024-03-24T11:23:32.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.ttl: 2592000 2024-03-24T11:23:32.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-24T11:23:32.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-24T11:23:32.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-24T11:23:32.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.enable_blob_files: false 2024-03-24T11:23:32.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.min_blob_size: 0 2024-03-24T11:23:32.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.blob_file_size: 268435456 2024-03-24T11:23:32.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-24T11:23:32.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-24T11:23:32.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-24T11:23:32.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-24T11:23:32.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-24T11:23:32.749 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.blob_file_starting_level: 0 2024-03-24T11:23:32.750 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-24T11:23:32.750 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 45.sst 2024-03-24T11:23:32.750 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-24T11:23:32.750 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 47, last_sequence is 22095, log_number is 43,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-24T11:23:32.750 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 43 2024-03-24T11:23:32.750 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 9fb791f4-da1a-43d7-9aee-0e7b374ab2d2 2024-03-24T11:23:32.750 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711279412477380, "job": 1, "event": "recovery_started", "wal_files": [43]} 2024-03-24T11:23:32.750 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #43 mode 2 2024-03-24T11:23:32.750 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711279412481748, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 48, "file_size": 772327, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 22093, "largest_seqno": 22807, "table_properties": {"data_size": 768739, "index_size": 1422, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 1157, "raw_key_size": 11265, "raw_average_key_size": 25, "raw_value_size": 760929, "raw_average_value_size": 1690, "num_data_blocks": 57, "num_entries": 450, "num_filter_entries": 450, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711279412, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "9fb791f4-da1a-43d7-9aee-0e7b374ab2d2", "db_session_id": "05XMXXZKHVVKY6A77WZ3", "orig_file_number": 48, "seqno_to_time_mapping": "N/A"}} 2024-03-24T11:23:32.750 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711279412481861, "job": 1, "event": "recovery_finished"} 2024-03-24T11:23:32.750 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: [db/version_set.cc:5047] Creating manifest 50 2024-03-24T11:23:32.750 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-24T11:23:32.750 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000043.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-24T11:23:32.751 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x563d010f8000 2024-03-24T11:23:32.751 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: DB pointer 0x563d010e2000 2024-03-24T11:23:32.751 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-24T11:23:32.751 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-24T11:23:32.751 INFO:journalctl@ceph.mon.a.smithi062.stdout: ** DB Stats ** 2024-03-24T11:23:32.751 INFO:journalctl@ceph.mon.a.smithi062.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-24T11:23:32.751 INFO:journalctl@ceph.mon.a.smithi062.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-24T11:23:32.751 INFO:journalctl@ceph.mon.a.smithi062.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-24T11:23:32.751 INFO:journalctl@ceph.mon.a.smithi062.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-24T11:23:32.751 INFO:journalctl@ceph.mon.a.smithi062.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-24T11:23:32.751 INFO:journalctl@ceph.mon.a.smithi062.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-24T11:23:32.751 INFO:journalctl@ceph.mon.a.smithi062.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-24T11:23:32.751 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2024-03-24T11:23:32.751 INFO:journalctl@ceph.mon.a.smithi062.stdout: ** Compaction Stats [default] ** 2024-03-24T11:23:32.751 INFO:journalctl@ceph.mon.a.smithi062.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-24T11:23:32.752 INFO:journalctl@ceph.mon.a.smithi062.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-24T11:23:32.752 INFO:journalctl@ceph.mon.a.smithi062.stdout: L0 1/0 754.23 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 291.0 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-03-24T11:23:32.752 INFO:journalctl@ceph.mon.a.smithi062.stdout: L6 1/0 10.19 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-03-24T11:23:32.752 INFO:journalctl@ceph.mon.a.smithi062.stdout: Sum 2/0 10.93 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 291.0 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-03-24T11:23:32.752 INFO:journalctl@ceph.mon.a.smithi062.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 291.0 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-03-24T11:23:32.752 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2024-03-24T11:23:32.752 INFO:journalctl@ceph.mon.a.smithi062.stdout: ** Compaction Stats [default] ** 2024-03-24T11:23:32.752 INFO:journalctl@ceph.mon.a.smithi062.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-24T11:23:32.752 INFO:journalctl@ceph.mon.a.smithi062.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-24T11:23:32.752 INFO:journalctl@ceph.mon.a.smithi062.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 291.0 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-03-24T11:23:32.753 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2024-03-24T11:23:32.753 INFO:journalctl@ceph.mon.a.smithi062.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-24T11:23:32.753 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2024-03-24T11:23:32.753 INFO:journalctl@ceph.mon.a.smithi062.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-24T11:23:32.753 INFO:journalctl@ceph.mon.a.smithi062.stdout: Flush(GB): cumulative 0.001, interval 0.001 2024-03-24T11:23:32.753 INFO:journalctl@ceph.mon.a.smithi062.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-24T11:23:32.753 INFO:journalctl@ceph.mon.a.smithi062.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-24T11:23:32.753 INFO:journalctl@ceph.mon.a.smithi062.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-24T11:23:32.753 INFO:journalctl@ceph.mon.a.smithi062.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-24T11:23:32.753 INFO:journalctl@ceph.mon.a.smithi062.stdout: Cumulative compaction: 0.00 GB write, 103.09 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-24T11:23:32.753 INFO:journalctl@ceph.mon.a.smithi062.stdout: Interval compaction: 0.00 GB write, 103.09 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-24T11:23:32.753 INFO:journalctl@ceph.mon.a.smithi062.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-24T11:23:32.753 INFO:journalctl@ceph.mon.a.smithi062.stdout: Block cache BinnedLRUCache@0x563d01007090#2 capacity: 512.00 MB usage: 2.86 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-03-24T11:23:32.754 INFO:journalctl@ceph.mon.a.smithi062.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,1.28 KB,0.000244379%) IndexBlock(1,1.58 KB,0.000301003%) Misc(1,0.00 KB,0%) 2024-03-24T11:23:32.754 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2024-03-24T11:23:32.754 INFO:journalctl@ceph.mon.a.smithi062.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-24T11:23:32.754 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: starting mon.a rank 0 at public addrs [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] at bind addrs [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid ebb89568-e9ce-11ee-95cd-87774f69a715 2024-03-24T11:23:32.754 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: mon.a@-1(???) e3 preinit fsid ebb89568-e9ce-11ee-95cd-87774f69a715 2024-03-24T11:23:32.754 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: mon.a@-1(???).mds e1 new map 2024-03-24T11:23:32.754 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: mon.a@-1(???).mds e1 print_map 2024-03-24T11:23:32.754 INFO:journalctl@ceph.mon.a.smithi062.stdout: e1 2024-03-24T11:23:32.754 INFO:journalctl@ceph.mon.a.smithi062.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T11:23:32.754 INFO:journalctl@ceph.mon.a.smithi062.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T11:23:32.754 INFO:journalctl@ceph.mon.a.smithi062.stdout: legacy client fscid: -1 2024-03-24T11:23:32.754 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2024-03-24T11:23:32.755 INFO:journalctl@ceph.mon.a.smithi062.stdout: No filesystems configured 2024-03-24T11:23:32.755 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: mon.a@-1(???).osd e107 crush map has features 3314933000854323200, adjusting msgr requires 2024-03-24T11:23:32.755 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: mon.a@-1(???).osd e107 crush map has features 432629239337189376, adjusting msgr requires 2024-03-24T11:23:32.755 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: mon.a@-1(???).osd e107 crush map has features 432629239337189376, adjusting msgr requires 2024-03-24T11:23:32.755 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: mon.a@-1(???).osd e107 crush map has features 432629239337189376, adjusting msgr requires 2024-03-24T11:23:32.755 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 ceph-mon[86786]: mon.a@-1(???).paxosservice(auth 1..24) refresh upgraded, format 0 -> 3 2024-03-24T11:23:32.755 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 bash[86766]: 37e940608c6af0a2b9b0cbd02a50be0676841f7673f003b5cc6431b002ee1d39 2024-03-24T11:23:32.755 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:32 smithi062 systemd[1]: Started Ceph mon.a for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:23:33.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:33 smithi062 ceph-mon[86786]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T11:23:33.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:33 smithi062 ceph-mon[86786]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T11:23:33.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:33 smithi062 ceph-mon[86786]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:23:33.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:33 smithi062 ceph-mon[86786]: pgmap v51: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:23:33.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:33 smithi062 ceph-mon[86786]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:23:33.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:33 smithi062 ceph-mon[86786]: pgmap v52: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:23:33.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:33 smithi062 ceph-mon[86786]: mon.a calling monitor election 2024-03-24T11:23:33.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:33 smithi062 ceph-mon[86786]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-24T11:23:33.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:33 smithi062 ceph-mon[86786]: monmap e3: 3 mons at {a=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0],b=[v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0],c=[v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-24T11:23:33.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:33 smithi062 ceph-mon[86786]: fsmap 2024-03-24T11:23:33.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:33 smithi062 ceph-mon[86786]: osdmap e107: 8 total, 8 up, 8 in 2024-03-24T11:23:33.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:33 smithi062 ceph-mon[86786]: mgrmap e37: y(active, since 92s), standbys: x 2024-03-24T11:23:33.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:33 smithi062 ceph-mon[86786]: overall HEALTH_OK 2024-03-24T11:23:33.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:33 smithi062 ceph-mon[86786]: from='mgr.25255 ' entity='' 2024-03-24T11:23:33.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:33 smithi062 ceph-mon[86786]: mgrmap e38: y(active, since 92s), standbys: x 2024-03-24T11:23:33.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:33 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T11:23:33.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:33 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T11:23:33.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:33 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:23:33.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:33 smithi062 ceph-mon[29666]: pgmap v51: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:23:33.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:33 smithi062 ceph-mon[29666]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:23:33.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:33 smithi062 ceph-mon[29666]: pgmap v52: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:23:33.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:33 smithi062 ceph-mon[29666]: mon.a calling monitor election 2024-03-24T11:23:33.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:33 smithi062 ceph-mon[29666]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-24T11:23:33.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:33 smithi062 ceph-mon[29666]: monmap e3: 3 mons at {a=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0],b=[v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0],c=[v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-24T11:23:33.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:33 smithi062 ceph-mon[29666]: fsmap 2024-03-24T11:23:33.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:33 smithi062 ceph-mon[29666]: osdmap e107: 8 total, 8 up, 8 in 2024-03-24T11:23:33.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:33 smithi062 ceph-mon[29666]: mgrmap e37: y(active, since 92s), standbys: x 2024-03-24T11:23:33.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:33 smithi062 ceph-mon[29666]: overall HEALTH_OK 2024-03-24T11:23:33.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:33 smithi062 ceph-mon[29666]: from='mgr.25255 ' entity='' 2024-03-24T11:23:33.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:33 smithi062 ceph-mon[29666]: mgrmap e38: y(active, since 92s), standbys: x 2024-03-24T11:23:34.354 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:23:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ignoring --setuser ceph since I am not root 2024-03-24T11:23:34.354 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:23:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ignoring --setgroup ceph since I am not root 2024-03-24T11:23:34.354 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:23:33 smithi062 ceph-mgr[69442]: -- 172.21.15.62:0/4226134569 <== mon.1 v2:172.21.15.62:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ffc7ae61a0 con 0x55ffc7abcc00 2024-03-24T11:23:34.358 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:33 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: ignoring --setuser ceph since I am not root 2024-03-24T11:23:34.358 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:33 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: ignoring --setgroup ceph since I am not root 2024-03-24T11:23:34.358 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:33 smithi139 ceph-mgr[73590]: -- 172.21.15.139:0/2214576440 <== mon.0 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55740b8701a0 con 0x55740b846800 2024-03-24T11:23:34.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:33 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T11:23:34.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:33 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T11:23:34.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:33 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:23:34.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:33 smithi139 ceph-mon[25719]: pgmap v51: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:23:34.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:33 smithi139 ceph-mon[25719]: from='mgr.25255 172.21.15.62:0/1510433132' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:23:34.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:33 smithi139 ceph-mon[25719]: pgmap v52: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:23:34.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:33 smithi139 ceph-mon[25719]: mon.a calling monitor election 2024-03-24T11:23:34.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:33 smithi139 ceph-mon[25719]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-24T11:23:34.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:33 smithi139 ceph-mon[25719]: monmap e3: 3 mons at {a=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0],b=[v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0],c=[v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-24T11:23:34.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:33 smithi139 ceph-mon[25719]: fsmap 2024-03-24T11:23:34.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:33 smithi139 ceph-mon[25719]: osdmap e107: 8 total, 8 up, 8 in 2024-03-24T11:23:34.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:33 smithi139 ceph-mon[25719]: mgrmap e37: y(active, since 92s), standbys: x 2024-03-24T11:23:34.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:33 smithi139 ceph-mon[25719]: overall HEALTH_OK 2024-03-24T11:23:34.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:33 smithi139 ceph-mon[25719]: from='mgr.25255 ' entity='' 2024-03-24T11:23:34.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:33 smithi139 ceph-mon[25719]: mgrmap e38: y(active, since 92s), standbys: x 2024-03-24T11:23:34.606 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:23:34 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:23:34.509+0000 7f4e948db200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-24T11:23:34.858 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:34 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:34.582+0000 7f630d373200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-24T11:23:34.858 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:34 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:34.674+0000 7f630d373200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-24T11:23:34.976 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:23:34 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:23:34.606+0000 7f4e948db200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-24T11:23:35.608 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:35 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:35.315+0000 7f630d373200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-24T11:23:35.608 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:35 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:35.409+0000 7f630d373200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-24T11:23:35.608 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:35 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:35.552+0000 7f630d373200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-24T11:23:35.613 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:23:35 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:23:35.274+0000 7f4e948db200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-24T11:23:35.613 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:23:35 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:23:35.370+0000 7f4e948db200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-24T11:23:35.613 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:23:35 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:23:35.514+0000 7f4e948db200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-24T11:23:35.613 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:23:35 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:23:35.612+0000 7f4e948db200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-24T11:23:35.926 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:23:35 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:23:35.698+0000 7f4e948db200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-24T11:23:35.926 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:35 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:35.647+0000 7f630d373200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-24T11:23:35.927 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:35 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:35.729+0000 7f630d373200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-24T11:23:36.191 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:35 smithi139 ceph-mon[25719]: mgrmap e39: y(active, since 93s), standbys: x 2024-03-24T11:23:36.191 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:35 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:35.994+0000 7f630d373200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-24T11:23:36.191 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:36 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:36.104+0000 7f630d373200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-24T11:23:36.191 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:36 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:36.190+0000 7f630d373200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-24T11:23:36.226 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:35 smithi062 ceph-mon[29666]: mgrmap e39: y(active, since 93s), standbys: x 2024-03-24T11:23:36.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:35 smithi062 ceph-mon[86786]: mgrmap e39: y(active, since 93s), standbys: x 2024-03-24T11:23:36.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:23:35 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:23:35.968+0000 7f4e948db200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-24T11:23:36.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:23:36 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:23:36.081+0000 7f4e948db200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-24T11:23:36.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:23:36 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:23:36.170+0000 7f4e948db200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-24T11:23:36.464 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:36 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:36.285+0000 7f630d373200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-24T11:23:36.727 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:23:36 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:23:36.268+0000 7f4e948db200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-24T11:23:36.727 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:23:36 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:23:36.448+0000 7f4e948db200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-24T11:23:36.857 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:36 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:36.463+0000 7f630d373200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-24T11:23:37.727 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:23:37 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:23:37.442+0000 7f4e948db200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-24T11:23:37.727 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:23:37 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:23:37.578+0000 7f4e948db200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-24T11:23:37.831 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:37 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:37.462+0000 7f630d373200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-24T11:23:37.831 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:37 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:37.598+0000 7f630d373200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-24T11:23:38.107 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:37 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:37.946+0000 7f630d373200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-24T11:23:38.108 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:38 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:38.037+0000 7f630d373200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-24T11:23:38.219 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:23:37 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:23:37.920+0000 7f4e948db200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-24T11:23:38.219 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:23:38 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:23:38.015+0000 7f4e948db200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-24T11:23:38.476 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:23:38 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:23:38.219+0000 7f4e948db200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-24T11:23:38.477 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:23:38 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:23:38.411+0000 7f4e948db200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-24T11:23:38.608 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:38 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:38.242+0000 7f630d373200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-24T11:23:38.608 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:38 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:38.434+0000 7f630d373200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-24T11:23:38.975 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:23:38 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:23:38.708+0000 7f4e948db200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-24T11:23:38.999 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:23:38 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:23:38.659Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"smithi062:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"172.21.15.139:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-24T11:23:38.999 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:38 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:38.730+0000 7f630d373200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-24T11:23:38.999 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:38 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:38.998+0000 7f630d373200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-24T11:23:39.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:23:38 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:23:38.975+0000 7f4e948db200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-24T11:23:39.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:23:39 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:23:39.222+0000 7f4e948db200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-24T11:23:39.357 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:39 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:39.238+0000 7f630d373200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-24T11:23:40.357 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:40 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:40.051+0000 7f630d373200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-24T11:23:40.476 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:23:40 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:23:40.076+0000 7f4e948db200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-24T11:23:40.953 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:40 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:40.625+0000 7f630d373200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-24T11:23:40.953 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:40 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:40.727+0000 7f630d373200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-24T11:23:40.953 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:40 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:40.951+0000 7f630d373200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-24T11:23:40.976 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:23:40 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:23:40.666+0000 7f4e948db200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-24T11:23:40.977 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:23:40 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:23:40.770+0000 7f4e948db200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-24T11:23:41.218 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:40 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: [24/Mar/2024:11:23:40] ENGINE Bus STARTING 2024-03-24T11:23:41.219 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:41 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: CherryPy Checker: 2024-03-24T11:23:41.219 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:41 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: The Application mounted at '' has an empty config. 2024-03-24T11:23:41.219 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:41 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: 2024-03-24T11:23:41.219 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:41 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: [24/Mar/2024:11:23:41] ENGINE Serving on http://:::9283 2024-03-24T11:23:41.219 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:23:41 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x[73586]: [24/Mar/2024:11:23:41] ENGINE Bus STARTED 2024-03-24T11:23:41.219 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:41 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-b[25697]: 2024-03-24T11:23:41.030+0000 7f2773a6e700 -1 mon.b@2(peon).osd e108 update_from_paxos full map CRC mismatch, resetting to canonical 2024-03-24T11:23:41.219 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:41 smithi139 ceph-mon[25719]: Standby manager daemon x restarted 2024-03-24T11:23:41.219 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:41 smithi139 ceph-mon[25719]: Standby manager daemon x started 2024-03-24T11:23:41.219 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:41 smithi139 ceph-mon[25719]: from='mgr.? 172.21.15.139:0/2587532481' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-24T11:23:41.219 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:41 smithi139 ceph-mon[25719]: from='mgr.? 172.21.15.139:0/2587532481' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T11:23:41.219 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:41 smithi139 ceph-mon[25719]: from='mgr.? 172.21.15.139:0/2587532481' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-24T11:23:41.220 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:41 smithi139 ceph-mon[25719]: from='mgr.? 172.21.15.139:0/2587532481' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T11:23:41.283 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:41 smithi062 ceph-mon[86786]: Standby manager daemon x restarted 2024-03-24T11:23:41.283 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:41 smithi062 ceph-mon[86786]: Standby manager daemon x started 2024-03-24T11:23:41.283 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:41 smithi062 ceph-mon[86786]: from='mgr.? 172.21.15.139:0/2587532481' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-24T11:23:41.283 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:41 smithi062 ceph-mon[86786]: from='mgr.? 172.21.15.139:0/2587532481' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T11:23:41.283 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:41 smithi062 ceph-mon[86786]: from='mgr.? 172.21.15.139:0/2587532481' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-24T11:23:41.283 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:41 smithi062 ceph-mon[86786]: from='mgr.? 172.21.15.139:0/2587532481' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T11:23:41.284 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:23:41 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:23:41.003+0000 7f4e948db200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-24T11:23:41.284 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:23:41 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: [24/Mar/2024:11:23:41] ENGINE Bus STARTING 2024-03-24T11:23:41.284 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:41 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-c[29661]: 2024-03-24T11:23:41.031+0000 7f3f927a1700 -1 mon.c@1(peon).osd e108 update_from_paxos full map CRC mismatch, resetting to canonical 2024-03-24T11:23:41.284 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:41 smithi062 ceph-mon[29666]: Standby manager daemon x restarted 2024-03-24T11:23:41.284 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:41 smithi062 ceph-mon[29666]: Standby manager daemon x started 2024-03-24T11:23:41.284 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:41 smithi062 ceph-mon[29666]: from='mgr.? 172.21.15.139:0/2587532481' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-24T11:23:41.284 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:41 smithi062 ceph-mon[29666]: from='mgr.? 172.21.15.139:0/2587532481' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T11:23:41.284 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:41 smithi062 ceph-mon[29666]: from='mgr.? 172.21.15.139:0/2587532481' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-24T11:23:41.285 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:41 smithi062 ceph-mon[29666]: from='mgr.? 172.21.15.139:0/2587532481' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T11:23:41.726 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:23:41 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: CherryPy Checker: 2024-03-24T11:23:41.727 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:23:41 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: The Application mounted at '' has an empty config. 2024-03-24T11:23:41.727 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:23:41 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:23:41.727 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:23:41 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: [24/Mar/2024:11:23:41] ENGINE Serving on http://:::9283 2024-03-24T11:23:41.727 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:23:41 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: [24/Mar/2024:11:23:41] ENGINE Bus STARTED 2024-03-24T11:23:42.074 INFO:teuthology.orchestra.run.smithi062.stdout:true 2024-03-24T11:23:42.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:42 smithi139 ceph-mon[25719]: Active manager daemon y restarted 2024-03-24T11:23:42.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:42 smithi139 ceph-mon[25719]: Activating manager daemon y 2024-03-24T11:23:42.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:42 smithi139 ceph-mon[25719]: mgrmap e40: y(active, since 99s), standbys: x 2024-03-24T11:23:42.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:42 smithi139 ceph-mon[25719]: osdmap e108: 8 total, 8 up, 8 in 2024-03-24T11:23:42.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:42 smithi139 ceph-mon[25719]: mgrmap e41: y(active, starting, since 0.010539s), standbys: x 2024-03-24T11:23:42.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:42 smithi139 ceph-mon[25719]: failed to encode map e108 with expected crc 2024-03-24T11:23:42.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:42 smithi139 ceph-mon[25719]: failed to encode map e108 with expected crc 2024-03-24T11:23:42.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T11:23:42.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T11:23:42.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:23:42.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-24T11:23:42.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-24T11:23:42.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T11:23:42.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T11:23:42.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T11:23:42.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T11:23:42.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T11:23:42.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T11:23:42.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T11:23:42.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T11:23:42.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-24T11:23:42.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-24T11:23:42.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-24T11:23:42.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:42 smithi139 ceph-mon[25719]: Manager daemon y is now available 2024-03-24T11:23:42.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:42 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:42.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:42.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi139", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:23:42.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:42 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi139", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:23:42.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi062", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:23:42.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:42 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi062", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:23:42.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:42.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:42.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:42 smithi139 ceph-mon[25719]: Updating smithi062:/etc/ceph/ceph.conf 2024-03-24T11:23:42.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:42 smithi139 ceph-mon[25719]: Updating smithi139:/etc/ceph/ceph.conf 2024-03-24T11:23:42.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:23:42.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-24T11:23:42.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:42 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-24T11:23:42.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:42 smithi139 ceph-mon[25719]: failed to encode map e108 with expected crc 2024-03-24T11:23:42.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:42 smithi139 ceph-mon[25719]: failed to encode map e108 with expected crc 2024-03-24T11:23:42.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-24T11:23:42.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:42 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-24T11:23:42.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:42 smithi139 ceph-mon[25719]: failed to encode map e108 with expected crc 2024-03-24T11:23:42.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:42 smithi139 ceph-mon[25719]: failed to encode map e108 with expected crc 2024-03-24T11:23:42.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:42 smithi139 ceph-mon[25719]: failed to encode map e108 with expected crc 2024-03-24T11:23:42.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:42 smithi139 ceph-mon[25719]: Updating smithi062:/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T11:23:42.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:42 smithi139 ceph-mon[25719]: Updating smithi139:/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T11:23:42.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:42 smithi139 ceph-mon[25719]: [24/Mar/2024:11:23:41] ENGINE Bus STARTING 2024-03-24T11:23:42.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:42 smithi139 ceph-mon[25719]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2024-03-24T11:23:42.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:42 smithi139 ceph-mon[25719]: Updating smithi139:/etc/ceph/ceph.client.admin.keyring 2024-03-24T11:23:42.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:42 smithi139 ceph-mon[25719]: [24/Mar/2024:11:23:41] ENGINE Serving on https://172.21.15.62:7151 2024-03-24T11:23:42.476 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:23:42.057+0000 7f4e4aad7700 -1 mgr.server handle_report got status from non-daemon mon.a 2024-03-24T11:23:42.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[86786]: Active manager daemon y restarted 2024-03-24T11:23:42.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[86786]: Activating manager daemon y 2024-03-24T11:23:42.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[86786]: mgrmap e40: y(active, since 99s), standbys: x 2024-03-24T11:23:42.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[86786]: osdmap e108: 8 total, 8 up, 8 in 2024-03-24T11:23:42.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[86786]: mgrmap e41: y(active, starting, since 0.010539s), standbys: x 2024-03-24T11:23:42.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[86786]: failed to encode map e108 with expected crc 2024-03-24T11:23:42.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[86786]: failed to encode map e108 with expected crc 2024-03-24T11:23:42.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T11:23:42.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T11:23:42.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:23:42.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-24T11:23:42.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-24T11:23:42.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T11:23:42.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T11:23:42.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T11:23:42.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T11:23:42.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T11:23:42.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T11:23:42.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T11:23:42.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T11:23:42.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-24T11:23:42.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-24T11:23:42.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-24T11:23:42.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[86786]: Manager daemon y is now available 2024-03-24T11:23:42.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:42.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:42.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi139", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:23:42.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi139", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:23:42.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi062", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:23:42.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[29666]: Active manager daemon y restarted 2024-03-24T11:23:42.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[29666]: Activating manager daemon y 2024-03-24T11:23:42.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[29666]: mgrmap e40: y(active, since 99s), standbys: x 2024-03-24T11:23:42.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[29666]: osdmap e108: 8 total, 8 up, 8 in 2024-03-24T11:23:42.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[29666]: mgrmap e41: y(active, starting, since 0.010539s), standbys: x 2024-03-24T11:23:42.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[29666]: failed to encode map e108 with expected crc 2024-03-24T11:23:42.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[29666]: failed to encode map e108 with expected crc 2024-03-24T11:23:42.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T11:23:42.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T11:23:42.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:23:42.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-24T11:23:42.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-24T11:23:42.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T11:23:42.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T11:23:42.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T11:23:42.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T11:23:42.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T11:23:42.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T11:23:42.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T11:23:42.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T11:23:42.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-24T11:23:42.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-24T11:23:42.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-24T11:23:42.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[29666]: Manager daemon y is now available 2024-03-24T11:23:42.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:42.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:42.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi139", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:23:42.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi139", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:23:42.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi062", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:23:42.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi062", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:23:42.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:42.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:42.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[86786]: Updating smithi062:/etc/ceph/ceph.conf 2024-03-24T11:23:42.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[86786]: Updating smithi139:/etc/ceph/ceph.conf 2024-03-24T11:23:42.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:23:42.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-24T11:23:42.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-24T11:23:42.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[86786]: failed to encode map e108 with expected crc 2024-03-24T11:23:42.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[86786]: failed to encode map e108 with expected crc 2024-03-24T11:23:42.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-24T11:23:42.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-24T11:23:42.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[86786]: failed to encode map e108 with expected crc 2024-03-24T11:23:42.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[86786]: failed to encode map e108 with expected crc 2024-03-24T11:23:42.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[86786]: failed to encode map e108 with expected crc 2024-03-24T11:23:42.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[86786]: Updating smithi062:/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T11:23:42.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[86786]: Updating smithi139:/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T11:23:42.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[86786]: [24/Mar/2024:11:23:41] ENGINE Bus STARTING 2024-03-24T11:23:42.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[86786]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2024-03-24T11:23:42.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[86786]: Updating smithi139:/etc/ceph/ceph.client.admin.keyring 2024-03-24T11:23:42.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[86786]: [24/Mar/2024:11:23:41] ENGINE Serving on https://172.21.15.62:7151 2024-03-24T11:23:42.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi062", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:23:42.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:42.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:42.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[29666]: Updating smithi062:/etc/ceph/ceph.conf 2024-03-24T11:23:42.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[29666]: Updating smithi139:/etc/ceph/ceph.conf 2024-03-24T11:23:42.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:23:42.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-24T11:23:42.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-24T11:23:42.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[29666]: failed to encode map e108 with expected crc 2024-03-24T11:23:42.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[29666]: failed to encode map e108 with expected crc 2024-03-24T11:23:42.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-24T11:23:42.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-24T11:23:42.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[29666]: failed to encode map e108 with expected crc 2024-03-24T11:23:42.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[29666]: failed to encode map e108 with expected crc 2024-03-24T11:23:42.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[29666]: failed to encode map e108 with expected crc 2024-03-24T11:23:42.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[29666]: Updating smithi062:/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T11:23:42.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[29666]: Updating smithi139:/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T11:23:42.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[29666]: [24/Mar/2024:11:23:41] ENGINE Bus STARTING 2024-03-24T11:23:42.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[29666]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2024-03-24T11:23:42.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[29666]: Updating smithi139:/etc/ceph/ceph.client.admin.keyring 2024-03-24T11:23:42.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:42 smithi062 ceph-mon[29666]: [24/Mar/2024:11:23:41] ENGINE Serving on https://172.21.15.62:7151 2024-03-24T11:23:42.766 INFO:teuthology.orchestra.run.smithi062.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T11:23:42.766 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi062 smithi062 running 12s ago 13m - - 2024-03-24T11:23:42.766 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi139 smithi139 running 16s ago 12m - - 2024-03-24T11:23:42.766 INFO:teuthology.orchestra.run.smithi062.stdout:alertmanager.a smithi062 *:9093,9094 running (2m) 12s ago 7m 25.1M - 0.23.0 ba2b418f427c 818298d58140 2024-03-24T11:23:42.766 INFO:teuthology.orchestra.run.smithi062.stdout:grafana.a smithi139 *:3000 running (2m) 16s ago 7m 48.0M - 8.3.5 dad864ee21e9 b48167e7171e 2024-03-24T11:23:42.766 INFO:teuthology.orchestra.run.smithi062.stdout:iscsi.foo.smithi062.dggsvk smithi062 running (76s) 12s ago 6m 50.6M - 3.5 e1d6a67b021e 0a98806ec839 2024-03-24T11:23:42.766 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.x smithi139 *:8443,9283,8765 running (27s) 16s ago 12m 447M - 19.0.0-1667-gdb0330b1 4f325f8440e9 d94987b11387 2024-03-24T11:23:42.766 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.y smithi062 *:8443,9283,8765 running (108s) 12s ago 14m 511M - 19.0.0-1667-gdb0330b1 4f325f8440e9 ca1a49028d65 2024-03-24T11:23:42.766 INFO:teuthology.orchestra.run.smithi062.stdout:mon.a smithi062 running (14m) 12s ago 14m 65.4M 2048M 17.2.0 e1d6a67b021e 712f01aa9180 2024-03-24T11:23:42.767 INFO:teuthology.orchestra.run.smithi062.stdout:mon.b smithi139 running (12m) 16s ago 12m 54.6M 2048M 17.2.0 e1d6a67b021e 8945cdb4e6aa 2024-03-24T11:23:42.767 INFO:teuthology.orchestra.run.smithi062.stdout:mon.c smithi062 running (12m) 12s ago 12m 44.7M 2048M 17.2.0 e1d6a67b021e 39110a16ab62 2024-03-24T11:23:42.767 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.a smithi062 *:9100 running (3m) 12s ago 8m 15.3M - 1.3.1 1dbe0e931976 79d5a02a999f 2024-03-24T11:23:42.767 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.b smithi139 *:9100 running (2m) 16s ago 8m 16.5M - 1.3.1 1dbe0e931976 8f9f4608e465 2024-03-24T11:23:42.767 INFO:teuthology.orchestra.run.smithi062.stdout:osd.0 smithi062 running (12m) 12s ago 12m 53.3M 4096M 17.2.0 e1d6a67b021e 8040b30d7170 2024-03-24T11:23:42.767 INFO:teuthology.orchestra.run.smithi062.stdout:osd.1 smithi062 running (11m) 12s ago 11m 55.0M 4096M 17.2.0 e1d6a67b021e e62d22e5dc29 2024-03-24T11:23:42.767 INFO:teuthology.orchestra.run.smithi062.stdout:osd.2 smithi062 running (11m) 12s ago 11m 48.5M 4096M 17.2.0 e1d6a67b021e 714f550103a7 2024-03-24T11:23:42.767 INFO:teuthology.orchestra.run.smithi062.stdout:osd.3 smithi062 running (10m) 12s ago 10m 52.7M 4096M 17.2.0 e1d6a67b021e 1d2868bd5195 2024-03-24T11:23:42.767 INFO:teuthology.orchestra.run.smithi062.stdout:osd.4 smithi139 running (10m) 16s ago 10m 57.3M 4096M 17.2.0 e1d6a67b021e 7568bfa9e74d 2024-03-24T11:23:42.767 INFO:teuthology.orchestra.run.smithi062.stdout:osd.5 smithi139 running (10m) 16s ago 10m 50.5M 4096M 17.2.0 e1d6a67b021e dd0abd599a14 2024-03-24T11:23:42.767 INFO:teuthology.orchestra.run.smithi062.stdout:osd.6 smithi139 running (9m) 16s ago 9m 50.7M 4096M 17.2.0 e1d6a67b021e a4b1289aca69 2024-03-24T11:23:42.767 INFO:teuthology.orchestra.run.smithi062.stdout:osd.7 smithi139 running (9m) 16s ago 9m 52.7M 4096M 17.2.0 e1d6a67b021e bc7fadbc1384 2024-03-24T11:23:42.767 INFO:teuthology.orchestra.run.smithi062.stdout:prometheus.a smithi139 *:9095 running (88s) 16s ago 8m 56.9M - 2.33.4 514e6a882f6e 2ed47f810d33 2024-03-24T11:23:42.767 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi062.sacpds smithi062 *:8000 running (6m) 12s ago 6m 72.5M - 17.2.0 e1d6a67b021e 89f4ac308e98 2024-03-24T11:23:42.767 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi139.yljesh smithi139 *:8000 running (6m) 16s ago 6m 71.4M - 17.2.0 e1d6a67b021e e189bdc12cae 2024-03-24T11:23:42.768 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi062.xvpxnk smithi062 *:80 running (6m) 12s ago 6m 72.3M - 17.2.0 e1d6a67b021e d1aa74adab44 2024-03-24T11:23:42.768 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi139.pbtrmp smithi139 *:80 running (6m) 16s ago 6m 74.9M - 17.2.0 e1d6a67b021e 7f8e336755d7 2024-03-24T11:23:43.182 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:23:43.182 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": { 2024-03-24T11:23:43.183 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 2, 2024-03-24T11:23:43.183 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 1 2024-03-24T11:23:43.183 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:23:43.183 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": { 2024-03-24T11:23:43.183 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-24T11:23:43.183 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:23:43.183 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": { 2024-03-24T11:23:43.183 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 8 2024-03-24T11:23:43.183 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:23:43.183 INFO:teuthology.orchestra.run.smithi062.stdout: "rgw": { 2024-03-24T11:23:43.183 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4 2024-03-24T11:23:43.183 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:23:43.183 INFO:teuthology.orchestra.run.smithi062.stdout: "overall": { 2024-03-24T11:23:43.184 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 14, 2024-03-24T11:23:43.184 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 3 2024-03-24T11:23:43.184 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-24T11:23:43.184 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:23:43.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:43 smithi139 ceph-mon[25719]: failed to encode map e108 with expected crc 2024-03-24T11:23:43.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:43 smithi139 ceph-mon[25719]: [24/Mar/2024:11:23:42] ENGINE Serving on http://172.21.15.62:8765 2024-03-24T11:23:43.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:43 smithi139 ceph-mon[25719]: [24/Mar/2024:11:23:42] ENGINE Bus STARTED 2024-03-24T11:23:43.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:43 smithi139 ceph-mon[25719]: mgrmap e42: y(active, since 1.02285s), standbys: x 2024-03-24T11:23:43.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:43 smithi139 ceph-mon[25719]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:23:43.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:43 smithi139 ceph-mon[25719]: from='client.25501 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:23:43.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:43 smithi139 ceph-mon[25719]: pgmap v3: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:23:43.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:43 smithi139 ceph-mon[25719]: Updating smithi139:/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T11:23:43.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:43 smithi139 ceph-mon[25719]: Updating smithi062:/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T11:23:43.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:43 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:43.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:43 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:43.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:43 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:43.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:43 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:43.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:43 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:43.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:43 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:43.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:43 smithi139 ceph-mon[25719]: from='client.25522 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:23:43.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi062", "caps": []}]: dispatch 2024-03-24T11:23:43.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:43 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi062", "caps": []}]: dispatch 2024-03-24T11:23:43.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:43 smithi139 ceph-mon[25719]: from='client.34143 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:23:43.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi139", "caps": []}]: dispatch 2024-03-24T11:23:43.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:43 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi139", "caps": []}]: dispatch 2024-03-24T11:23:43.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:43 smithi062 ceph-mon[86786]: failed to encode map e108 with expected crc 2024-03-24T11:23:43.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:43 smithi062 ceph-mon[86786]: [24/Mar/2024:11:23:42] ENGINE Serving on http://172.21.15.62:8765 2024-03-24T11:23:43.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:43 smithi062 ceph-mon[86786]: [24/Mar/2024:11:23:42] ENGINE Bus STARTED 2024-03-24T11:23:43.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:43 smithi062 ceph-mon[86786]: mgrmap e42: y(active, since 1.02285s), standbys: x 2024-03-24T11:23:43.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:43 smithi062 ceph-mon[86786]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:23:43.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:43 smithi062 ceph-mon[86786]: from='client.25501 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:23:43.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:43 smithi062 ceph-mon[86786]: pgmap v3: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:23:43.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:43 smithi062 ceph-mon[86786]: Updating smithi139:/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T11:23:43.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:43 smithi062 ceph-mon[86786]: Updating smithi062:/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T11:23:43.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:43 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:43.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:43 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:43.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:43 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:43.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:43 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:43.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:43 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:43.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:43 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:43.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:43 smithi062 ceph-mon[86786]: from='client.25522 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:23:43.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:43 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi062", "caps": []}]: dispatch 2024-03-24T11:23:43.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:43 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi062", "caps": []}]: dispatch 2024-03-24T11:23:43.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:43 smithi062 ceph-mon[86786]: from='client.34143 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:23:43.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:43 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi139", "caps": []}]: dispatch 2024-03-24T11:23:43.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:43 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi139", "caps": []}]: dispatch 2024-03-24T11:23:43.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:43 smithi062 ceph-mon[29666]: failed to encode map e108 with expected crc 2024-03-24T11:23:43.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:43 smithi062 ceph-mon[29666]: [24/Mar/2024:11:23:42] ENGINE Serving on http://172.21.15.62:8765 2024-03-24T11:23:43.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:43 smithi062 ceph-mon[29666]: [24/Mar/2024:11:23:42] ENGINE Bus STARTED 2024-03-24T11:23:43.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:43 smithi062 ceph-mon[29666]: mgrmap e42: y(active, since 1.02285s), standbys: x 2024-03-24T11:23:43.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:43 smithi062 ceph-mon[29666]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:23:43.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:43 smithi062 ceph-mon[29666]: from='client.25501 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:23:43.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:43 smithi062 ceph-mon[29666]: pgmap v3: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:23:43.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:43 smithi062 ceph-mon[29666]: Updating smithi139:/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T11:23:43.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:43 smithi062 ceph-mon[29666]: Updating smithi062:/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T11:23:43.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:43 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:43.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:43 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:43.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:43 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:43.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:43 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:43.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:43 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:43.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:43 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:43.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:43 smithi062 ceph-mon[29666]: from='client.25522 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:23:43.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:43 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi062", "caps": []}]: dispatch 2024-03-24T11:23:43.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:43 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi062", "caps": []}]: dispatch 2024-03-24T11:23:43.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:43 smithi062 ceph-mon[29666]: from='client.34143 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:23:43.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:43 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi139", "caps": []}]: dispatch 2024-03-24T11:23:43.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:43 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi139", "caps": []}]: dispatch 2024-03-24T11:23:43.547 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:23:43.547 INFO:teuthology.orchestra.run.smithi062.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69", 2024-03-24T11:23:43.547 INFO:teuthology.orchestra.run.smithi062.stdout: "in_progress": true, 2024-03-24T11:23:43.547 INFO:teuthology.orchestra.run.smithi062.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T11:23:43.547 INFO:teuthology.orchestra.run.smithi062.stdout: "services_complete": [ 2024-03-24T11:23:43.547 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr" 2024-03-24T11:23:43.547 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-03-24T11:23:43.547 INFO:teuthology.orchestra.run.smithi062.stdout: "progress": "2/23 daemons upgraded", 2024-03-24T11:23:43.547 INFO:teuthology.orchestra.run.smithi062.stdout: "message": "", 2024-03-24T11:23:43.547 INFO:teuthology.orchestra.run.smithi062.stdout: "is_paused": false 2024-03-24T11:23:43.547 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:23:44.163 INFO:teuthology.orchestra.run.smithi062.stdout:HEALTH_OK 2024-03-24T11:23:44.164 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:44 smithi062 ceph-mon[86786]: pgmap v4: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:23:44.164 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:44 smithi062 ceph-mon[86786]: from='client.? 172.21.15.62:0/82953553' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:44.164 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:44 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:44.164 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:44 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:44.164 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:44 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi062", "caps": []}]: dispatch 2024-03-24T11:23:44.164 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:44 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi062", "caps": []}]: dispatch 2024-03-24T11:23:44.164 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:44 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi139", "caps": []}]: dispatch 2024-03-24T11:23:44.164 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:44 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi139", "caps": []}]: dispatch 2024-03-24T11:23:44.164 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:44 smithi062 ceph-mon[86786]: mgrmap e43: y(active, since 2s), standbys: x 2024-03-24T11:23:44.164 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:44 smithi062 ceph-mon[86786]: from='client.34161 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:23:44.164 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:44 smithi062 ceph-mon[29666]: pgmap v4: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:23:44.164 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:44 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/82953553' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:44.165 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:44 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:44.165 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:44 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:44.165 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:44 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi062", "caps": []}]: dispatch 2024-03-24T11:23:44.165 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:44 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi062", "caps": []}]: dispatch 2024-03-24T11:23:44.165 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:44 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi139", "caps": []}]: dispatch 2024-03-24T11:23:44.165 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:44 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi139", "caps": []}]: dispatch 2024-03-24T11:23:44.165 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:44 smithi062 ceph-mon[29666]: mgrmap e43: y(active, since 2s), standbys: x 2024-03-24T11:23:44.165 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:44 smithi062 ceph-mon[29666]: from='client.34161 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:23:44.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:44 smithi139 ceph-mon[25719]: pgmap v4: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:23:44.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:44 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/82953553' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:44.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:44.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:44.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi062", "caps": []}]: dispatch 2024-03-24T11:23:44.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:44 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi062", "caps": []}]: dispatch 2024-03-24T11:23:44.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi139", "caps": []}]: dispatch 2024-03-24T11:23:44.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:44 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi139", "caps": []}]: dispatch 2024-03-24T11:23:44.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:44 smithi139 ceph-mon[25719]: mgrmap e43: y(active, since 2s), standbys: x 2024-03-24T11:23:44.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:44 smithi139 ceph-mon[25719]: from='client.34161 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:23:45.217 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:45 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/1348497946' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:23:45.217 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:45 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:45.217 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:45 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:45.217 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:45 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:45.217 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:45 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:45.217 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:45 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:45.217 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:45 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:45.217 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:45 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:45.217 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:45 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:45.404 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:45 smithi062 ceph-mon[86786]: from='client.? 172.21.15.62:0/1348497946' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:23:45.404 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:45 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:45.404 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:45 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:45.404 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:45 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:45.404 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:45 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:45.404 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:45 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:45.404 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:45 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:45.405 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:45 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:45.405 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:45 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:45.405 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:45 smithi062 ceph-mon[29666]: from='client.? 172.21.15.62:0/1348497946' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:23:45.405 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:45 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:45.405 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:45 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:45.405 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:45 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:45.405 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:45 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:45.405 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:45 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:45.405 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:45 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:45.405 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:45 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:45.405 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:45 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:46.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[86786]: pgmap v5: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:23:46.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:46.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:46.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:46.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:46.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[86786]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:46.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:46.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:46.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:46.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:46.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:46.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[86786]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:46.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[86786]: mgrmap e44: y(active, since 4s), standbys: x 2024-03-24T11:23:46.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:46.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:46.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:46.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:46.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:46.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[86786]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:46.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:46.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:46.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:46.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:46.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:46.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[86786]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:46.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:46.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:46.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:46.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:46.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:46.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:46.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[29666]: pgmap v5: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:23:46.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:46.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:46.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:46.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:46.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:46.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:46.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:46.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:46.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:46.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:46.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:46.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[29666]: mgrmap e44: y(active, since 4s), standbys: x 2024-03-24T11:23:46.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:46.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:46.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:46.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:46.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:46.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:46.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:46.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:46.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:46.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:46.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:46.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:46.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:46.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:46.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:46.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:46.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:46.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:46 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:46.538 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:46 smithi139 ceph-mon[25719]: pgmap v5: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:23:46.538 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:46 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:46.538 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:46.538 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:46.538 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:46 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:46.538 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:46 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:46.538 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:46.538 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:46.539 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:46.539 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:46.539 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:46 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:46.539 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:46 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:46.539 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:46 smithi139 ceph-mon[25719]: mgrmap e44: y(active, since 4s), standbys: x 2024-03-24T11:23:46.539 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:46.539 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:46.539 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:46.539 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:46.539 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:46 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:46.539 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:46 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:46.539 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:46.539 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:46.539 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:46.539 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:46.540 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:46 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:46.540 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:46 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:46.540 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:46 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:46.540 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:46.540 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:46.540 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:46.540 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:46.540 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:46 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:47.448 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:47.448 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:47.448 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:47.448 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:47.448 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:47.448 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:47.448 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:47.448 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:47.448 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:47.448 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:47.449 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:47.449 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:47.449 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:47.449 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:47.449 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:47.449 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:47.449 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:47.449 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:47.449 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:47.449 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:47.449 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:47.449 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:47.449 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:47.449 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:47.450 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:47.450 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:47.450 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:47.450 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:47.450 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:47.450 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:47.450 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:47.450 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:47.450 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:47.450 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:47.450 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:47.450 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:47.450 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:47.451 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:47.451 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:47.451 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:47.451 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:47.451 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:47.451 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:47.451 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:47.451 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:47.451 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:47.451 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:47.451 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:47.451 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:47.451 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:47.451 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:47.452 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:47.452 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:47.452 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:47.452 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:47 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:47.600 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:47.601 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:47.601 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:47.601 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:47.601 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:47.601 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:47.601 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:47.601 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:47.601 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:47.601 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:47.601 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:47.602 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:47.602 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:47.602 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:47.602 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:47.602 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:47.602 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:47.602 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:47.602 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:47.602 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:47.602 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:47.602 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:47.602 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:47.602 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:47.602 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:47.603 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:47.603 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:47.603 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:47.603 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:47.603 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:47.603 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:47.603 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:47.603 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:47.603 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:47.603 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:47.603 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:47.603 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:47.603 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:47.603 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:47.603 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:47.604 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:47.604 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:47.604 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:47.604 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:47.604 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:47.604 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:47.604 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:47.604 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:47.604 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:47.604 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:47.604 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:47.604 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:47.604 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:47.605 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:47.605 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:47.605 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:47.605 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:47.605 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:47.605 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:47.605 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:47.605 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:47.606 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:47.606 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:47.606 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:47.606 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:47.606 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:47.606 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:47.606 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:47.606 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:47.606 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:47.606 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:47.606 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:47.606 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:47.606 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:47.607 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:47.607 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:47.607 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:47.607 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:47.607 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:47.607 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:47.607 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:47.607 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:47.607 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:47.607 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:47.607 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:47.607 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:47.607 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:47.608 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:47.608 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:47.608 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:47.608 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:47.608 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:47.608 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:47.608 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:47.608 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:47.608 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:47.608 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:47.608 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:47.608 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:47.608 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:47.608 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:47.608 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:47.609 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:47.609 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:47.609 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:47.609 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:47.609 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:47.609 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:47.610 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:47.610 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:47 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:48.507 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: pgmap v6: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:23:48.507 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:48.507 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:48.507 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:48.507 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:48.507 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:48.508 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:48.508 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:48.508 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:48.508 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:48.508 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:48.508 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:48.508 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:48.508 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:48.508 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:48.508 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:48.508 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:48.508 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:48.509 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:48.509 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:48.509 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:48.509 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:48.509 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:48.509 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:48.509 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:48.509 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:48.509 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:48.509 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:48.509 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:48.510 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:48.510 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:48.510 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:48.510 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:48.510 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:48.510 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:48.510 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:48.510 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:48.510 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:48.510 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:48.510 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:48.510 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:48.511 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:48.511 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:48.511 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:48.511 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:48.511 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:48.511 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:48.511 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:48.511 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:48.511 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:48.511 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:48.511 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:48.512 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:48.512 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:48 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:48.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: pgmap v6: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:23:48.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:48.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:48.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:48.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:48.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:48.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:48.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:48.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:48.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:48.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:48.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:48.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:48.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:48.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:48.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:48.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:48.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:48.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:48.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:48.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:48.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:48.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:48.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:48.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:48.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:48.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:48.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:48.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:48.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:48.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: pgmap v6: 161 pgs: 161 active+clean; 585 KiB data, 96 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:23:48.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:48.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:48.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:48.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:48.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:48.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:48.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:48.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:48.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:48.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:48.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:48.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:48.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:48.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:48.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:48.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:48.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:48.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:48.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:48.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:48.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:48.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:48.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:48.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:48.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:48.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:48.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:48.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:48.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:48.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:48.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:48.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:48.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:48.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:48.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:48.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:48.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:48.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:48.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:48.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:48.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:48.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:48.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:48.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:48.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:48.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:48.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:48.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:48.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:48.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:48.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:48.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:48.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:48.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:48.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:48.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:48.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:48.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:48.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:48.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:48.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:48.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:48.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:48.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:48.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:48.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:48.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:48.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:48.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:48.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:48.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:48.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:48.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:48.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:48.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:48.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:48.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:48 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:49.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:49 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:49.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:49 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:49.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:49 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:49.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:49 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:49.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:49 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:49.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:49 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:49.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:49 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:49.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:49 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:49.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:49 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:49.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:49 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:49.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:49 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:49.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:49 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:49.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:49 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:49.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:49 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:49.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:49 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:49.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:49 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:49.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:49 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:49.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:49 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:49.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:49 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-24T11:23:49.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:49 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mon ok-to-stop", "ids": ["a"]}]: dispatch 2024-03-24T11:23:49.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:49 smithi139 ceph-mon[25719]: Upgrade: It appears safe to stop mon.a 2024-03-24T11:23:49.398 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:49 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:49.398 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:49 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:49.398 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:49 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:49.398 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:49 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:49.398 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:49 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:49.398 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:49 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:49.398 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:49 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:49.398 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:49 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:49.398 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:49 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:49.399 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:49 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:49.399 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:49 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:49.399 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:49 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:49.399 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:49 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:49.399 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:49 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:49.399 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:49 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:49.399 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:49 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:49.399 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:49 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:49.399 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:49 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:49.399 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:49 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-24T11:23:49.399 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:49 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mon ok-to-stop", "ids": ["a"]}]: dispatch 2024-03-24T11:23:49.399 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:49 smithi062 ceph-mon[29666]: Upgrade: It appears safe to stop mon.a 2024-03-24T11:23:49.399 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:49 smithi062 ceph-mon[86786]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:49.400 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:49 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:49.400 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:49 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:49.400 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:49 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:49.400 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:49 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:49.400 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:49 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:49.400 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:49 smithi062 ceph-mon[86786]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:49.400 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:49 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:49.400 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:49 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:49.400 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:49 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:49.400 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:49 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:23:49.400 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:49 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:49.400 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:49 smithi062 ceph-mon[86786]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:23:49.400 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:49 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:49.400 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:49 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:49.401 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:49 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:49.401 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:49 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:23:49.401 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:49 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:23:49.401 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:49 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-24T11:23:49.401 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:49 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mon ok-to-stop", "ids": ["a"]}]: dispatch 2024-03-24T11:23:49.401 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:49 smithi062 ceph-mon[86786]: Upgrade: It appears safe to stop mon.a 2024-03-24T11:23:50.657 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:50 smithi062 ceph-mon[29666]: pgmap v7: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-03-24T11:23:50.658 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:50 smithi062 ceph-mon[86786]: pgmap v7: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-03-24T11:23:50.816 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:50 smithi139 ceph-mon[25719]: pgmap v7: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-03-24T11:23:50.976 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:23:50 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:23:50] "GET /metrics HTTP/1.1" 200 35520 "" "Prometheus/2.33.4" 2024-03-24T11:23:51.918 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:51 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:51.918 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:51 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:51.918 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:51 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:51.918 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:51 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:51.918 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:51 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:51.918 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:51 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:51.918 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:51 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T11:23:51.918 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:51 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T11:23:51.918 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:51 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:51.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:51 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:51.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:51 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:51.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:51 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:51.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:51 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:51.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:51 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:51.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:51 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:51.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:51 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T11:23:51.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:51 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T11:23:51.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:51 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:51.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:51 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:51.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:51 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:51.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:51 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:51.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:51 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:51.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:51 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:51.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:51 smithi062 ceph-mon[86786]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:23:51.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:51 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T11:23:51.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:51 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T11:23:51.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:51 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:23:52.943 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:52 smithi062 ceph-mon[86786]: pgmap v8: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2024-03-24T11:23:52.943 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:52 smithi062 ceph-mon[86786]: Upgrade: Updating mon.a 2024-03-24T11:23:52.943 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:52 smithi062 ceph-mon[86786]: Deploying daemon mon.a on smithi062 2024-03-24T11:23:52.943 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:52 smithi062 ceph-mon[29666]: pgmap v8: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2024-03-24T11:23:52.943 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:52 smithi062 ceph-mon[29666]: Upgrade: Updating mon.a 2024-03-24T11:23:52.943 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:52 smithi062 ceph-mon[29666]: Deploying daemon mon.a on smithi062 2024-03-24T11:23:53.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:52 smithi139 ceph-mon[25719]: pgmap v8: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2024-03-24T11:23:53.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:52 smithi139 ceph-mon[25719]: Upgrade: Updating mon.a 2024-03-24T11:23:53.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:52 smithi139 ceph-mon[25719]: Deploying daemon mon.a on smithi062 2024-03-24T11:23:53.857 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:53 smithi139 ceph-mon[25719]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:23:53.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:53 smithi062 ceph-mon[29666]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:23:53.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:53 smithi062 ceph-mon[86786]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:23:54.926 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:54 smithi062 ceph-mon[86786]: pgmap v9: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-24T11:23:54.927 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:54 smithi062 ceph-mon[29666]: pgmap v9: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-24T11:23:55.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:54 smithi139 ceph-mon[25719]: pgmap v9: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-24T11:23:56.717 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:56 smithi062 ceph-mon[29666]: pgmap v10: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-24T11:23:56.717 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:23:56 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:23:56.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:56 smithi062 ceph-mon[86786]: pgmap v10: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-24T11:23:56.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:56 smithi062 ceph-mon[86786]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:23:56.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:56 smithi062 systemd[1]: Stopping Ceph mon.a for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:23:57.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:56 smithi139 ceph-mon[25719]: pgmap v10: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-24T11:23:57.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:23:56 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:23:57.459 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:57 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-a[86782]: 2024-03-24T11:23:57.188+0000 7f7492299700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T11:23:57.459 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:57 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-a[86782]: 2024-03-24T11:23:57.188+0000 7f7492299700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-24T11:23:57.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:57 smithi062 podman[90251]: 2024-03-24 11:23:57.459204697 +0000 UTC m=+0.386727754 container died 37e940608c6af0a2b9b0cbd02a50be0676841f7673f003b5cc6431b002ee1d39 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-a, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-db0330b, CEPH_POINT_RELEASE=) 2024-03-24T11:23:58.062 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:57 smithi062 podman[90251]: 2024-03-24 11:23:57.950207427 +0000 UTC m=+0.877730485 container cleanup 37e940608c6af0a2b9b0cbd02a50be0676841f7673f003b5cc6431b002ee1d39 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-db0330b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-24T11:23:58.063 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:57 smithi062 bash[90251]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-a 2024-03-24T11:23:58.313 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:58 smithi062 podman[90279]: 2024-03-24 11:23:58.168437759 +0000 UTC m=+0.934645550 container remove 37e940608c6af0a2b9b0cbd02a50be0676841f7673f003b5cc6431b002ee1d39 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-a, org.label-schema.license=GPLv2, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, io.buildah.version=1.33.5) 2024-03-24T11:23:58.673 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:58 smithi062 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-24T11:23:58.674 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:58 smithi062 systemd[1]: Stopped Ceph mon.a for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:23:58.674 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:58 smithi062 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mon.a.service: Consumed 1.578s CPU time. 2024-03-24T11:23:58.674 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:58 smithi062 systemd[1]: Starting Ceph mon.a for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:23:58.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:58 smithi062 podman[90512]: 2024-03-24 11:23:58.672635761 +0000 UTC m=+0.024095427 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:23:58.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:58 smithi062 podman[90512]: 2024-03-24 11:23:58.794455691 +0000 UTC m=+0.145915355 container create f7afe6e28c88760a16d29c35f0ce7746b2d2e57a5739780da2869d121bf30c0f (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-a, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-24T11:23:59.108 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:23:58 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:23:58.659Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"smithi062:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"172.21.15.139:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-24T11:23:59.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 podman[90512]: 2024-03-24 11:23:59.064470204 +0000 UTC m=+0.415929884 container init f7afe6e28c88760a16d29c35f0ce7746b2d2e57a5739780da2869d121bf30c0f (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-a, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-db0330b, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T11:23:59.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 podman[90512]: 2024-03-24 11:23:59.067630143 +0000 UTC m=+0.419089809 container start f7afe6e28c88760a16d29c35f0ce7746b2d2e57a5739780da2869d121bf30c0f (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-a, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, GIT_CLEAN=True) 2024-03-24T11:23:59.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: set uid:gid to 167:167 (ceph:ceph) 2024-03-24T11:23:59.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev), process ceph-mon, pid 2 2024-03-24T11:23:59.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: pidfile_write: ignore empty --pid-file 2024-03-24T11:23:59.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: load: jerasure load: lrc 2024-03-24T11:23:59.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: RocksDB version: 7.9.2 2024-03-24T11:23:59.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Git sha 0 2024-03-24T11:23:59.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Compile date 2024-03-16 10:36:11 2024-03-24T11:23:59.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: DB SUMMARY 2024-03-24T11:23:59.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: DB Session ID: P1NG98K58P31MS0GIK15 2024-03-24T11:23:59.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: CURRENT file: CURRENT 2024-03-24T11:23:59.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: IDENTITY file: IDENTITY 2024-03-24T11:23:59.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: MANIFEST file: MANIFEST-000050 size: 277 Bytes 2024-03-24T11:23:59.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 2, files: 000045.sst 000048.sst 2024-03-24T11:23:59.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000049.log size: 2663446 ; 2024-03-24T11:23:59.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.error_if_exists: 0 2024-03-24T11:23:59.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.create_if_missing: 0 2024-03-24T11:23:59.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.paranoid_checks: 1 2024-03-24T11:23:59.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-24T11:23:59.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-24T11:23:59.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-24T11:23:59.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.env: 0x55bbeedf0ae0 2024-03-24T11:23:59.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.fs: PosixFileSystem 2024-03-24T11:23:59.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.info_log: 0x55bbf08c78c0 2024-03-24T11:23:59.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.max_file_opening_threads: 16 2024-03-24T11:23:59.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.statistics: (nil) 2024-03-24T11:23:59.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.use_fsync: 0 2024-03-24T11:23:59.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.max_log_file_size: 0 2024-03-24T11:23:59.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-24T11:23:59.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-24T11:23:59.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.keep_log_file_num: 1000 2024-03-24T11:23:59.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.recycle_log_file_num: 0 2024-03-24T11:23:59.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.allow_fallocate: 1 2024-03-24T11:23:59.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.allow_mmap_reads: 0 2024-03-24T11:23:59.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.allow_mmap_writes: 0 2024-03-24T11:23:59.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.use_direct_reads: 0 2024-03-24T11:23:59.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-24T11:23:59.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.create_missing_column_families: 0 2024-03-24T11:23:59.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.db_log_dir: 2024-03-24T11:23:59.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.wal_dir: 2024-03-24T11:23:59.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-24T11:23:59.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-24T11:23:59.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-24T11:23:59.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-24T11:23:59.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-24T11:23:59.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-24T11:23:59.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.advise_random_on_open: 1 2024-03-24T11:23:59.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.db_write_buffer_size: 0 2024-03-24T11:23:59.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.write_buffer_manager: 0x55bbf089adc0 2024-03-24T11:23:59.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-24T11:23:59.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-24T11:23:59.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-24T11:23:59.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.rate_limiter: (nil) 2024-03-24T11:23:59.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-24T11:23:59.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.wal_recovery_mode: 2 2024-03-24T11:23:59.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.enable_thread_tracking: 0 2024-03-24T11:23:59.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.enable_pipelined_write: 0 2024-03-24T11:23:59.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.unordered_write: 0 2024-03-24T11:23:59.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-24T11:23:59.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-24T11:23:59.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-24T11:23:59.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-24T11:23:59.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.row_cache: None 2024-03-24T11:23:59.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.wal_filter: None 2024-03-24T11:23:59.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-24T11:23:59.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.allow_ingest_behind: 0 2024-03-24T11:23:59.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.two_write_queues: 0 2024-03-24T11:23:59.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.manual_wal_flush: 0 2024-03-24T11:23:59.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.wal_compression: 0 2024-03-24T11:23:59.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.atomic_flush: 0 2024-03-24T11:23:59.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-24T11:23:59.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-24T11:23:59.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-24T11:23:59.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.log_readahead_size: 0 2024-03-24T11:23:59.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-24T11:23:59.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.best_efforts_recovery: 0 2024-03-24T11:23:59.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-24T11:23:59.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-24T11:23:59.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.allow_data_in_errors: 0 2024-03-24T11:23:59.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.db_host_id: __hostname__ 2024-03-24T11:23:59.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-24T11:23:59.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.max_background_jobs: 2 2024-03-24T11:23:59.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.max_background_compactions: -1 2024-03-24T11:23:59.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.max_subcompactions: 1 2024-03-24T11:23:59.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-24T11:23:59.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-24T11:23:59.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-24T11:23:59.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.max_total_wal_size: 0 2024-03-24T11:23:59.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-24T11:23:59.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-24T11:23:59.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-24T11:23:59.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-24T11:23:59.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.max_open_files: -1 2024-03-24T11:23:59.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.bytes_per_sync: 0 2024-03-24T11:23:59.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-24T11:23:59.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-24T11:23:59.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.compaction_readahead_size: 0 2024-03-24T11:23:59.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.max_background_flushes: -1 2024-03-24T11:23:59.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Compression algorithms supported: 2024-03-24T11:23:59.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-24T11:23:59.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: kZSTD supported: 0 2024-03-24T11:23:59.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: kXpressCompression supported: 0 2024-03-24T11:23:59.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: kLZ4HCCompression supported: 1 2024-03-24T11:23:59.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: kZlibCompression supported: 1 2024-03-24T11:23:59.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: kSnappyCompression supported: 1 2024-03-24T11:23:59.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: kLZ4Compression supported: 1 2024-03-24T11:23:59.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: kBZip2Compression supported: 0 2024-03-24T11:23:59.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-24T11:23:59.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-24T11:23:59.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000050 2024-03-24T11:23:59.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-24T11:23:59.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-24T11:23:59.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.merge_operator: 2024-03-24T11:23:59.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.compaction_filter: None 2024-03-24T11:23:59.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.compaction_filter_factory: None 2024-03-24T11:23:59.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.sst_partitioner_factory: None 2024-03-24T11:23:59.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-24T11:23:59.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-24T11:23:59.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55bbf08c79c0) 2024-03-24T11:23:59.487 INFO:journalctl@ceph.mon.a.smithi062.stdout: cache_index_and_filter_blocks: 1 2024-03-24T11:23:59.487 INFO:journalctl@ceph.mon.a.smithi062.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-24T11:23:59.487 INFO:journalctl@ceph.mon.a.smithi062.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-24T11:23:59.487 INFO:journalctl@ceph.mon.a.smithi062.stdout: pin_top_level_index_and_filter: 1 2024-03-24T11:23:59.487 INFO:journalctl@ceph.mon.a.smithi062.stdout: index_type: 0 2024-03-24T11:23:59.488 INFO:journalctl@ceph.mon.a.smithi062.stdout: data_block_index_type: 0 2024-03-24T11:23:59.488 INFO:journalctl@ceph.mon.a.smithi062.stdout: index_shortening: 1 2024-03-24T11:23:59.488 INFO:journalctl@ceph.mon.a.smithi062.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-24T11:23:59.488 INFO:journalctl@ceph.mon.a.smithi062.stdout: checksum: 4 2024-03-24T11:23:59.488 INFO:journalctl@ceph.mon.a.smithi062.stdout: no_block_cache: 0 2024-03-24T11:23:59.488 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_cache: 0x55bbf08cb090 2024-03-24T11:23:59.488 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_cache_name: BinnedLRUCache 2024-03-24T11:23:59.488 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_cache_options: 2024-03-24T11:23:59.488 INFO:journalctl@ceph.mon.a.smithi062.stdout: capacity : 536870912 2024-03-24T11:23:59.488 INFO:journalctl@ceph.mon.a.smithi062.stdout: num_shard_bits : 4 2024-03-24T11:23:59.488 INFO:journalctl@ceph.mon.a.smithi062.stdout: strict_capacity_limit : 0 2024-03-24T11:23:59.488 INFO:journalctl@ceph.mon.a.smithi062.stdout: high_pri_pool_ratio: 0.000 2024-03-24T11:23:59.488 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_cache_compressed: (nil) 2024-03-24T11:23:59.488 INFO:journalctl@ceph.mon.a.smithi062.stdout: persistent_cache: (nil) 2024-03-24T11:23:59.489 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_size: 4096 2024-03-24T11:23:59.489 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_size_deviation: 10 2024-03-24T11:23:59.489 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_restart_interval: 16 2024-03-24T11:23:59.489 INFO:journalctl@ceph.mon.a.smithi062.stdout: index_block_restart_interval: 1 2024-03-24T11:23:59.489 INFO:journalctl@ceph.mon.a.smithi062.stdout: metadata_block_size: 4096 2024-03-24T11:23:59.489 INFO:journalctl@ceph.mon.a.smithi062.stdout: partition_filters: 0 2024-03-24T11:23:59.489 INFO:journalctl@ceph.mon.a.smithi062.stdout: use_delta_encoding: 1 2024-03-24T11:23:59.489 INFO:journalctl@ceph.mon.a.smithi062.stdout: filter_policy: bloomfilter 2024-03-24T11:23:59.489 INFO:journalctl@ceph.mon.a.smithi062.stdout: whole_key_filtering: 1 2024-03-24T11:23:59.489 INFO:journalctl@ceph.mon.a.smithi062.stdout: verify_compression: 0 2024-03-24T11:23:59.489 INFO:journalctl@ceph.mon.a.smithi062.stdout: read_amp_bytes_per_bit: 0 2024-03-24T11:23:59.489 INFO:journalctl@ceph.mon.a.smithi062.stdout: format_version: 5 2024-03-24T11:23:59.489 INFO:journalctl@ceph.mon.a.smithi062.stdout: enable_index_compression: 1 2024-03-24T11:23:59.489 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_align: 0 2024-03-24T11:23:59.489 INFO:journalctl@ceph.mon.a.smithi062.stdout: max_auto_readahead_size: 262144 2024-03-24T11:23:59.490 INFO:journalctl@ceph.mon.a.smithi062.stdout: prepopulate_block_cache: 0 2024-03-24T11:23:59.490 INFO:journalctl@ceph.mon.a.smithi062.stdout: initial_auto_readahead_size: 8192 2024-03-24T11:23:59.490 INFO:journalctl@ceph.mon.a.smithi062.stdout: num_file_reads_for_auto_readahead: 2 2024-03-24T11:23:59.490 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.write_buffer_size: 33554432 2024-03-24T11:23:59.490 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.max_write_buffer_number: 2 2024-03-24T11:23:59.490 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.compression: NoCompression 2024-03-24T11:23:59.490 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.bottommost_compression: Disabled 2024-03-24T11:23:59.490 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.prefix_extractor: nullptr 2024-03-24T11:23:59.490 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-24T11:23:59.490 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.num_levels: 7 2024-03-24T11:23:59.490 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-24T11:23:59.490 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-24T11:23:59.490 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-24T11:23:59.490 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-24T11:23:59.491 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-24T11:23:59.491 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-24T11:23:59.491 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-24T11:23:59.491 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-24T11:23:59.491 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-24T11:23:59.491 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-24T11:23:59.491 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-24T11:23:59.491 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-24T11:23:59.491 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-24T11:23:59.491 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.compression_opts.level: 32767 2024-03-24T11:23:59.491 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.compression_opts.strategy: 0 2024-03-24T11:23:59.491 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-24T11:23:59.491 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-24T11:23:59.491 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-24T11:23:59.491 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-24T11:23:59.492 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.compression_opts.enabled: false 2024-03-24T11:23:59.492 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-24T11:23:59.492 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-24T11:23:59.492 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-24T11:23:59.492 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-24T11:23:59.492 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.target_file_size_base: 67108864 2024-03-24T11:23:59.492 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-24T11:23:59.492 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-24T11:23:59.492 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-24T11:23:59.492 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-24T11:23:59.492 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-24T11:23:59.492 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-24T11:23:59.492 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-24T11:23:59.492 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-24T11:23:59.493 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-24T11:23:59.493 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-24T11:23:59.493 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-24T11:23:59.493 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-24T11:23:59.493 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-24T11:23:59.493 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-24T11:23:59.493 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.arena_block_size: 1048576 2024-03-24T11:23:59.493 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-24T11:23:59.493 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-24T11:23:59.493 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.disable_auto_compactions: 0 2024-03-24T11:23:59.493 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-24T11:23:59.493 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-24T11:23:59.493 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-24T11:23:59.493 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-24T11:23:59.493 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-24T11:23:59.494 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-24T11:23:59.494 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-24T11:23:59.494 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-24T11:23:59.494 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-24T11:23:59.494 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-24T11:23:59.494 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.table_properties_collectors: 2024-03-24T11:23:59.494 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.inplace_update_support: 0 2024-03-24T11:23:59.494 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-24T11:23:59.494 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-24T11:23:59.494 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-24T11:23:59.494 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-24T11:23:59.494 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.bloom_locality: 0 2024-03-24T11:23:59.494 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.max_successive_merges: 0 2024-03-24T11:23:59.494 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-24T11:23:59.495 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.paranoid_file_checks: 0 2024-03-24T11:23:59.495 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.force_consistency_checks: 1 2024-03-24T11:23:59.495 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.report_bg_io_stats: 0 2024-03-24T11:23:59.495 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.ttl: 2592000 2024-03-24T11:23:59.495 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-24T11:23:59.495 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-24T11:23:59.495 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-24T11:23:59.495 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.enable_blob_files: false 2024-03-24T11:23:59.495 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.min_blob_size: 0 2024-03-24T11:23:59.495 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.blob_file_size: 268435456 2024-03-24T11:23:59.495 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-24T11:23:59.495 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-24T11:23:59.495 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-24T11:23:59.495 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-24T11:23:59.495 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-24T11:23:59.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.blob_file_starting_level: 0 2024-03-24T11:23:59.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-24T11:23:59.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 45.sst 2024-03-24T11:23:59.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-24T11:23:59.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000050 succeeded,manifest_file_number is 50, next_file_number is 52, last_sequence is 22807, log_number is 44,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 44 2024-03-24T11:23:59.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 44 2024-03-24T11:23:59.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 9fb791f4-da1a-43d7-9aee-0e7b374ab2d2 2024-03-24T11:23:59.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711279439097745, "job": 1, "event": "recovery_started", "wal_files": [49]} 2024-03-24T11:23:59.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #49 mode 2 2024-03-24T11:23:59.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711279439105052, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 53, "file_size": 2521876, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 22810, "largest_seqno": 23771, "table_properties": {"data_size": 2516655, "index_size": 2858, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 1349, "raw_key_size": 12177, "raw_average_key_size": 23, "raw_value_size": 2506297, "raw_average_value_size": 4876, "num_data_blocks": 128, "num_entries": 514, "num_filter_entries": 514, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711279439, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "9fb791f4-da1a-43d7-9aee-0e7b374ab2d2", "db_session_id": "P1NG98K58P31MS0GIK15", "orig_file_number": 53, "seqno_to_time_mapping": "N/A"}} 2024-03-24T11:23:59.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711279439105161, "job": 1, "event": "recovery_finished"} 2024-03-24T11:23:59.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: [db/version_set.cc:5047] Creating manifest 55 2024-03-24T11:23:59.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-24T11:23:59.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000049.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-24T11:23:59.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55bbf09bc000 2024-03-24T11:23:59.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: DB pointer 0x55bbf09a6000 2024-03-24T11:23:59.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-24T11:23:59.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-24T11:23:59.497 INFO:journalctl@ceph.mon.a.smithi062.stdout: ** DB Stats ** 2024-03-24T11:23:59.497 INFO:journalctl@ceph.mon.a.smithi062.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-24T11:23:59.497 INFO:journalctl@ceph.mon.a.smithi062.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-24T11:23:59.497 INFO:journalctl@ceph.mon.a.smithi062.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-24T11:23:59.497 INFO:journalctl@ceph.mon.a.smithi062.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-24T11:23:59.497 INFO:journalctl@ceph.mon.a.smithi062.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-24T11:23:59.497 INFO:journalctl@ceph.mon.a.smithi062.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-24T11:23:59.497 INFO:journalctl@ceph.mon.a.smithi062.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-24T11:23:59.497 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2024-03-24T11:23:59.497 INFO:journalctl@ceph.mon.a.smithi062.stdout: ** Compaction Stats [default] ** 2024-03-24T11:23:59.498 INFO:journalctl@ceph.mon.a.smithi062.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-24T11:23:59.498 INFO:journalctl@ceph.mon.a.smithi062.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-24T11:23:59.498 INFO:journalctl@ceph.mon.a.smithi062.stdout: L0 2/0 3.14 MB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 644.3 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-03-24T11:23:59.498 INFO:journalctl@ceph.mon.a.smithi062.stdout: L6 1/0 10.19 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-03-24T11:23:59.498 INFO:journalctl@ceph.mon.a.smithi062.stdout: Sum 3/0 13.33 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 644.3 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-03-24T11:23:59.498 INFO:journalctl@ceph.mon.a.smithi062.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 644.3 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-03-24T11:23:59.498 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2024-03-24T11:23:59.498 INFO:journalctl@ceph.mon.a.smithi062.stdout: ** Compaction Stats [default] ** 2024-03-24T11:23:59.498 INFO:journalctl@ceph.mon.a.smithi062.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-24T11:23:59.498 INFO:journalctl@ceph.mon.a.smithi062.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-24T11:23:59.498 INFO:journalctl@ceph.mon.a.smithi062.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 644.3 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-03-24T11:23:59.498 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2024-03-24T11:23:59.498 INFO:journalctl@ceph.mon.a.smithi062.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-24T11:23:59.498 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2024-03-24T11:23:59.498 INFO:journalctl@ceph.mon.a.smithi062.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-24T11:23:59.499 INFO:journalctl@ceph.mon.a.smithi062.stdout: Flush(GB): cumulative 0.002, interval 0.002 2024-03-24T11:23:59.499 INFO:journalctl@ceph.mon.a.smithi062.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-24T11:23:59.499 INFO:journalctl@ceph.mon.a.smithi062.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-24T11:23:59.499 INFO:journalctl@ceph.mon.a.smithi062.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-24T11:23:59.499 INFO:journalctl@ceph.mon.a.smithi062.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-24T11:23:59.499 INFO:journalctl@ceph.mon.a.smithi062.stdout: Cumulative compaction: 0.00 GB write, 214.53 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-24T11:23:59.499 INFO:journalctl@ceph.mon.a.smithi062.stdout: Interval compaction: 0.00 GB write, 214.53 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-24T11:23:59.499 INFO:journalctl@ceph.mon.a.smithi062.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-24T11:23:59.499 INFO:journalctl@ceph.mon.a.smithi062.stdout: Block cache BinnedLRUCache@0x55bbf08cb090#2 capacity: 512.00 MB usage: 7.34 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.7e-05 secs_since: 0 2024-03-24T11:23:59.499 INFO:journalctl@ceph.mon.a.smithi062.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,2.69 KB,0.0005126%) IndexBlock(2,4.66 KB,0.000888109%) Misc(1,0.00 KB,0%) 2024-03-24T11:23:59.499 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2024-03-24T11:23:59.499 INFO:journalctl@ceph.mon.a.smithi062.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-24T11:23:59.499 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: starting mon.a rank 0 at public addrs [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] at bind addrs [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid ebb89568-e9ce-11ee-95cd-87774f69a715 2024-03-24T11:23:59.499 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: mon.a@-1(???) e3 preinit fsid ebb89568-e9ce-11ee-95cd-87774f69a715 2024-03-24T11:23:59.500 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: mon.a@-1(???).mds e1 new map 2024-03-24T11:23:59.500 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: mon.a@-1(???).mds e1 print_map 2024-03-24T11:23:59.500 INFO:journalctl@ceph.mon.a.smithi062.stdout: e1 2024-03-24T11:23:59.500 INFO:journalctl@ceph.mon.a.smithi062.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T11:23:59.500 INFO:journalctl@ceph.mon.a.smithi062.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T11:23:59.500 INFO:journalctl@ceph.mon.a.smithi062.stdout: legacy client fscid: -1 2024-03-24T11:23:59.500 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2024-03-24T11:23:59.500 INFO:journalctl@ceph.mon.a.smithi062.stdout: No filesystems configured 2024-03-24T11:23:59.500 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: mon.a@-1(???).osd e108 crush map has features 3314933000854323200, adjusting msgr requires 2024-03-24T11:23:59.500 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: mon.a@-1(???).osd e108 crush map has features 432629239337189376, adjusting msgr requires 2024-03-24T11:23:59.500 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: mon.a@-1(???).osd e108 crush map has features 432629239337189376, adjusting msgr requires 2024-03-24T11:23:59.500 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: mon.a@-1(???).osd e108 crush map has features 432629239337189376, adjusting msgr requires 2024-03-24T11:23:59.500 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 ceph-mon[90553]: mon.a@-1(???).paxosservice(auth 1..25) refresh upgraded, format 0 -> 3 2024-03-24T11:23:59.500 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 bash[90512]: f7afe6e28c88760a16d29c35f0ce7746b2d2e57a5739780da2869d121bf30c0f 2024-03-24T11:23:59.501 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:23:59 smithi062 systemd[1]: Started Ceph mon.a for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:24:00.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:00 smithi062 ceph-mon[29666]: pgmap v11: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-24T11:24:00.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:00 smithi062 ceph-mon[29666]: pgmap v12: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-24T11:24:00.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:00 smithi062 ceph-mon[29666]: mon.a calling monitor election 2024-03-24T11:24:00.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:00 smithi062 ceph-mon[29666]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-24T11:24:00.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:00 smithi062 ceph-mon[29666]: monmap e3: 3 mons at {a=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0],b=[v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0],c=[v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-24T11:24:00.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:00 smithi062 ceph-mon[29666]: fsmap 2024-03-24T11:24:00.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:00 smithi062 ceph-mon[29666]: osdmap e108: 8 total, 8 up, 8 in 2024-03-24T11:24:00.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:00 smithi062 ceph-mon[29666]: mgrmap e44: y(active, since 19s), standbys: x 2024-03-24T11:24:00.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:00 smithi062 ceph-mon[29666]: overall HEALTH_OK 2024-03-24T11:24:00.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:00 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='' 2024-03-24T11:24:00.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:00 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='' 2024-03-24T11:24:00.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:00 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:00.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:00 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:00.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:00 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:00.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:00 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:00.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:00 smithi062 ceph-mon[90553]: pgmap v11: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-24T11:24:00.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:00 smithi062 ceph-mon[90553]: pgmap v12: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-24T11:24:00.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:00 smithi062 ceph-mon[90553]: mon.a calling monitor election 2024-03-24T11:24:00.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:00 smithi062 ceph-mon[90553]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-24T11:24:00.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:00 smithi062 ceph-mon[90553]: monmap e3: 3 mons at {a=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0],b=[v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0],c=[v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-24T11:24:00.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:00 smithi062 ceph-mon[90553]: fsmap 2024-03-24T11:24:00.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:00 smithi062 ceph-mon[90553]: osdmap e108: 8 total, 8 up, 8 in 2024-03-24T11:24:00.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:00 smithi062 ceph-mon[90553]: mgrmap e44: y(active, since 19s), standbys: x 2024-03-24T11:24:00.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:00 smithi062 ceph-mon[90553]: overall HEALTH_OK 2024-03-24T11:24:00.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:00 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='' 2024-03-24T11:24:00.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:00 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='' 2024-03-24T11:24:00.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:00 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:00.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:00.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:00.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:00 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:00.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:00 smithi139 ceph-mon[25719]: pgmap v11: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-24T11:24:00.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:00 smithi139 ceph-mon[25719]: pgmap v12: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-24T11:24:00.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:00 smithi139 ceph-mon[25719]: mon.a calling monitor election 2024-03-24T11:24:00.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:00 smithi139 ceph-mon[25719]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-24T11:24:00.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:00 smithi139 ceph-mon[25719]: monmap e3: 3 mons at {a=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0],b=[v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0],c=[v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-24T11:24:00.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:00 smithi139 ceph-mon[25719]: fsmap 2024-03-24T11:24:00.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:00 smithi139 ceph-mon[25719]: osdmap e108: 8 total, 8 up, 8 in 2024-03-24T11:24:00.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:00 smithi139 ceph-mon[25719]: mgrmap e44: y(active, since 19s), standbys: x 2024-03-24T11:24:00.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:00 smithi139 ceph-mon[25719]: overall HEALTH_OK 2024-03-24T11:24:00.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:00 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='' 2024-03-24T11:24:00.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:00 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='' 2024-03-24T11:24:00.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:00 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:00.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:00 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:00.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:00 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:00.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:00 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:01.226 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:24:00 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:24:00] "GET /metrics HTTP/1.1" 200 38319 "" "Prometheus/2.33.4" 2024-03-24T11:24:01.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:01.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:01.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:01.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:01.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:01.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:01.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:01.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:01.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:01.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:01.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:01.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:01.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:01.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:01.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:01.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:01.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:01.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:01.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:01.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:01.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:01.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:01.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:01.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:01.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:01.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:01.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:01.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:01.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:01.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:01.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:01.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:01.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:01.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:01.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:01.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:01.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:01.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:01.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:01.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:01.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:01.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:01.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:01.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:01.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:01.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:01.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:01.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:01.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:01.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:01.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:01.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:01.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:01.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:01.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:01.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:01.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:01.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:01.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:01.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:01.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:01.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:01.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:01.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:01.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:01.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:01.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:01.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:01.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:01.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:01.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:01.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:01.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:01.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:01.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:01.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:01.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:01.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:01 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:01.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:01 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:01.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:01 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:01.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:01 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:01.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:01 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:01.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:01 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:01.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:01 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:01.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:01 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:01.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:01 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:01.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:01 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:01.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:01 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:01.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:01 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:01.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:01 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:01.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:01 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:01.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:01 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:01.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:01 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:01.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:01 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:01.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:01 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:01.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:01 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:01.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:01 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:01.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:01 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:01.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:01 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:01.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:01 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:01.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:01 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:01.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:01 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:01.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:01 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:01.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:01 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:01.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:01 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:01.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:01 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:01.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:01 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:01.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:01 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:01.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:01 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:01.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:01 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:01.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:01 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:01.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:01 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:01.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:01 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:01.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:01 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:01.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:01 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:01.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:01 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:01.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:01 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:02.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:02.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: pgmap v13: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:24:02.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:02.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:02.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:02.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:02.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:02.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:02.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:02.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:02.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:02.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:02.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:02.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:02.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:02.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:02.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:02.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:02.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:02.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:02.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:02.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:02.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:02.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:02.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:02.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:02.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:02.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:02.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:02.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:02.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:02.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:02.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:02.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:02.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:02.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:02.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:02.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:02.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:02.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:02.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:02.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:02.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:02.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:02.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:02.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:02.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:02.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:02.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:02.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:02.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:02.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:02.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:02.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:02.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:02.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:02.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:02.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:02.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:02.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:02.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:02.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:02.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:02.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:02 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:02.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:02.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: pgmap v13: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:24:02.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:02.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:02.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:02.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:02.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:02.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:02.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:02.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:02.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:02.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:02.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:02.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:02.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:02.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:02.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:02.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:02.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:02.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:02.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:02.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:02.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:02.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:02.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:02.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:02.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:02.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:02.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:02.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:02.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:02.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:02.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: pgmap v13: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:24:02.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:02.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:02.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:02.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:02.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:02.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:02.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:02.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:02.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:02.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:02.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:02.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:02.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:02.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:02.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:02.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:02.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:02.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:02.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:02.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:02.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:02.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:02.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:02.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:02.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:02.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:02.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:02.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:02.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:02.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:02.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:02.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:02.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:02.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:02.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:02.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:02.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:02.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:02.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:02.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:02.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:02.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:02.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:02.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:02.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:02.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:02.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:02.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:02.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:02.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:02.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:02.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:02.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:02.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:02.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:02.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:02.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:02.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:02.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:02.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:02.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:02.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:02.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:02.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:02.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:02.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:02.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:02.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:02.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:02.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:02.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:02.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:02.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:02.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:02.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:02.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:02.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:02.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:02.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:02.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:02.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:02.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:02.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:02.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:02.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:02.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:02.989 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:02.989 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:02.989 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:02.989 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:02.989 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:02.989 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:02.989 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:02.989 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:02.989 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:02 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:03.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:03.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:24:03.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:03.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:03.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:03.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:03.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:03.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:03.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:03.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:03.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:03.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:03.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:03.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:03.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:03.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:03.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:03.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:03.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:03.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:03.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:03.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:03.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:03.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:03.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:03.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:03.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:03.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:03.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:03.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:03.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:03.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:03.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:24:03.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:03.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:03.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:03.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:03.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:03.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:03.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:03.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:03.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:03.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:03.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:03.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:03.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:03.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:03.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:03.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:03.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:03.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:03.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:03.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:03.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:03.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:03.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:03.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:03.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:03.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:03.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:03.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:03.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:03.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:03.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:03.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:03.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:03.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:03.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:03.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:03.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:03.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:03.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:03.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:03.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:03.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:03.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:03.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:03.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:03.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:03.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:03.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:03.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:03.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:03.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:03.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:03.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:03.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:03.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:03.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:03.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:03.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:03.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:03.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:03.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:03.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:03.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:03.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:03.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:03.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:03.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:03.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:03.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:03.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:03.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:03.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:03.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:03.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:03.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:03.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:03.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:03.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:03.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:03.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:03.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:03.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:03.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:03.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:03.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:03.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:03.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:03.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:03.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:03.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:03.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:03.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:03.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:03.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:03.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:03.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:03.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:03.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:03.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:03.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:03.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:03.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:03.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:03.989 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:03.989 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:03.989 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:03.989 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:03.989 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:03.989 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:03.989 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:03.989 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:03 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:04.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:04.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:24:04.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:04.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:04.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:04.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:04.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:04.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:04.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:04.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:04.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:04.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:04.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:04.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:04.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:04.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:04.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:04.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:04.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:04.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:04.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:04.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:04.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:04.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:04.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:04.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:04.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:04.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:04.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:04.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:04.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:04.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:04.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:04.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:04.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:04.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:04.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:04.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:04.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:04.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:04.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:04.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:04.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:04.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:04.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:04.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:04.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:04.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:04.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:04.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:04.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:04.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:04.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:04.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:04.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:04.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:04.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:04.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:04.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:04.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:04.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:04.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:04.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:04.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:04.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:04.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:04.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:04.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:04.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:04.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:04.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:04.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:03 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:04.888 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:04.888 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: pgmap v14: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:24:04.889 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:04.889 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:04.889 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:04.889 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:04.889 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:04.889 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:04.889 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:04.889 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:04.889 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:04.889 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:04.889 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:04.889 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:04.889 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:04.889 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:04.890 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:04.890 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:04.890 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:04.890 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:04.890 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:04.890 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:04.890 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:04.890 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:04.890 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:04.890 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:04.890 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:04.890 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:04.890 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:04.890 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:04.891 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:04.891 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:04.891 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: pgmap v14: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:24:04.891 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:04.891 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:04.891 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:04.891 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:04.891 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:04.891 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:04.891 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:04.891 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:04.892 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:04.892 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:04.892 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:04.892 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:04.892 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:04.892 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:04.892 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:04.892 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:04.892 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:04.892 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:04.892 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:04.892 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:04.892 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:04.892 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:04.893 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:04.893 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:04.893 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:04.893 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:04.893 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:04.893 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:04.893 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:04.893 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:04.893 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:04.893 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:04.893 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:04.893 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:04.894 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:04.894 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:04.894 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:04.894 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:04.894 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:04.894 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:04.894 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:04.894 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:04.894 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:04.894 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:04.894 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:04.894 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:04.894 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:04.895 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:04.895 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:04.895 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:04.895 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:04.895 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:04.895 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:04.895 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:04.895 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:04.895 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:04.895 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:04.895 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:04.895 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:04.896 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:04.896 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:04.896 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:04.896 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:04.896 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:04.896 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:04.896 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:04.896 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:04.896 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:04.896 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:04.896 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:04.896 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:04.896 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:04.896 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:04.897 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:04.897 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:04.897 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:04.897 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:04.897 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:04.897 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:04.897 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:04.897 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:04.897 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:04.897 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:04.897 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:04.897 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:04.897 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:04.897 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:04.898 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:04.898 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:04.898 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:04.898 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:04.898 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:04.898 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:04 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:05.010 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:05.010 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: pgmap v14: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:24:05.010 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:05.010 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:05.010 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:05.010 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:05.010 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:05.010 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:05.010 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:05.010 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:05.010 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:05.010 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:05.011 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:05.011 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:05.011 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:05.011 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:05.011 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:05.011 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:05.011 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:05.011 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:05.011 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:05.011 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:05.011 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:05.011 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:05.011 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:05.012 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:05.012 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:05.012 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:05.012 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:05.012 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:05.012 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:05.012 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:05.012 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:05.013 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:05.013 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:05.013 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:05.013 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:05.013 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:05.013 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:05.013 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:05.013 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:05.013 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:05.013 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:05.013 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:05.013 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:05.013 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:05.013 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:05.014 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:05.014 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:05.014 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:05.014 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:05.014 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:05.014 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:05.014 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:05.014 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:05.014 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:05.014 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:05.014 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:05.014 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:05.014 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:05.015 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:05.015 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:05.015 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:04 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:05.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:05.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:05.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:05.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:05.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:05.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:05.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:05.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:05.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:05.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:05.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:05.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:05.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:05.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:05.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:05.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:05.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:05.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:05.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:05.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:05.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:05.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:05.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:05.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:05.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:05.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:05.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:05.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:05.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:05.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:05.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:05.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:05.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:05.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:05.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:05.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:05.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:05.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:05.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:05.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:05.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:05.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:05.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:05.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:05.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:05.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:05.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:05.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:05.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:05.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:05.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:05.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:05.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:05.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:05.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:05.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:05.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:05.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:05.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:05.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:05.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:05.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:05.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:05.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:05.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:05.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:05.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:05.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:05.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:05.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:05.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:05.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:05.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:05.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:05.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:05.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:05.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:05.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:05.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:05.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:05.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:05.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:05.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:05.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:05.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:05.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:05.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:05.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:05.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:05.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:05.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:05.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:05.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:05.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:05.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:05.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:05.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:05.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:05.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:05.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:05.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:05.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:05.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:05.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:05.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:05.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:05 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:06.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:06.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:06.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:06.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:06.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:06.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:06.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:06.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:06.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:06.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:06.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:06.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:06.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:06.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:06.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:06.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:06.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:06.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:06.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:06.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:06.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:06.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:06.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:06.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:06.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:06.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:06.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:06.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:06.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:06.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:06.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:06.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:06.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:06.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:06.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:06.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:06.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:06.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:06.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:06.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:06.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:06.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:06.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:06.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:06.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:06.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:06.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:06.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:06.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:06.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:06.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:06.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:06.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:05 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:06.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:06 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:06.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:06 smithi062 ceph-mon[29666]: pgmap v15: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:24:06.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:06 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:06.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:06 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:06.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:06 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:06.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:06 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:06.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:06 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:06.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:06 smithi062 ceph-mon[29666]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:06.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:06 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:06.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:06 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:06.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:06 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:06.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:06 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:06.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:06 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:06.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:06 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-24T11:24:06.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:06 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mon ok-to-stop", "ids": ["c"]}]: dispatch 2024-03-24T11:24:06.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:06 smithi062 ceph-mon[29666]: Upgrade: It appears safe to stop mon.c 2024-03-24T11:24:06.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:06 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:06.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:06 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:06.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:06 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:06.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:06 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:06.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:06 smithi062 ceph-mon[90553]: pgmap v15: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:24:06.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:06 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:06.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:06 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:06.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:06 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:06.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:06 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:06.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:06 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:06.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:06 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:06.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:06 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:06.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:06 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:06.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:06 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:06.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:06.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:06.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-24T11:24:06.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mon ok-to-stop", "ids": ["c"]}]: dispatch 2024-03-24T11:24:06.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:06 smithi062 ceph-mon[90553]: Upgrade: It appears safe to stop mon.c 2024-03-24T11:24:06.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:06 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:06.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:06 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:06.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:06 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:07.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:06 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:07.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:06 smithi139 ceph-mon[25719]: pgmap v15: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:24:07.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:06 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:07.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:06 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:07.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:06 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:07.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:06 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:07.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:06 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:07.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:06 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:07.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:06 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:07.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:06 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:07.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:06 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:07.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:06 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:07.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:06 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:07.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:06 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-24T11:24:07.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:06 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mon ok-to-stop", "ids": ["c"]}]: dispatch 2024-03-24T11:24:07.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:06 smithi139 ceph-mon[25719]: Upgrade: It appears safe to stop mon.c 2024-03-24T11:24:07.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:06 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:07.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:06 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:07.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:06 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:08.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:08 smithi062 ceph-mon[90553]: pgmap v16: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:24:08.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:08 smithi062 ceph-mon[29666]: pgmap v16: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:24:09.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:08 smithi139 ceph-mon[25719]: pgmap v16: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:24:09.108 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:24:08 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:24:08.659Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"smithi062:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"172.21.15.139:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-24T11:24:10.338 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:10 smithi062 ceph-mon[90553]: pgmap v17: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:24:10.339 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:10 smithi062 ceph-mon[90553]: Upgrade: Updating mon.c 2024-03-24T11:24:10.339 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:10 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:10.339 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T11:24:10.339 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T11:24:10.339 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:10.339 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:10 smithi062 ceph-mon[90553]: Deploying daemon mon.c on smithi062 2024-03-24T11:24:10.339 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:10 smithi062 ceph-mon[29666]: pgmap v17: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:24:10.340 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:10 smithi062 ceph-mon[29666]: Upgrade: Updating mon.c 2024-03-24T11:24:10.340 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:10 smithi062 ceph-mon[29666]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:10.340 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:10 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T11:24:10.340 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:10 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T11:24:10.340 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:10 smithi062 ceph-mon[29666]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:10.340 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:10 smithi062 ceph-mon[29666]: Deploying daemon mon.c on smithi062 2024-03-24T11:24:10.580 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:10 smithi139 ceph-mon[25719]: pgmap v17: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:24:10.580 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:10 smithi139 ceph-mon[25719]: Upgrade: Updating mon.c 2024-03-24T11:24:10.580 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:10 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:10.580 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:10 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T11:24:10.580 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:10 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T11:24:10.580 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:10 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:10.580 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:10 smithi139 ceph-mon[25719]: Deploying daemon mon.c on smithi062 2024-03-24T11:24:11.061 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:24:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:24:10] "GET /metrics HTTP/1.1" 200 38319 "" "Prometheus/2.33.4" 2024-03-24T11:24:11.587 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:11 smithi062 systemd[1]: Stopping Ceph mon.c for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:24:11.587 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:11 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-c[29661]: 2024-03-24T11:24:11.574+0000 7f3f97fac700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T11:24:11.587 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:11 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-c[29661]: 2024-03-24T11:24:11.574+0000 7f3f97fac700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-03-24T11:24:11.934 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:11 smithi062 podman[93432]: 2024-03-24 11:24:11.646425446 +0000 UTC m=+0.173514383 container died 39110a16ab6274f3f5d12019071044375633567274f6ac19b2c1cc44a62ccb2c (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-c, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, io.openshift.expose-services=, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, io.k8s.display-name=CentOS Stream 8, com.redhat.component=centos-stream-container, maintainer=Guillaume Abrioux , description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, build-date=2022-05-03T08:36:31.336870, CEPH_POINT_RELEASE=-17.2.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.openshift.tags=base centos centos-stream, release=754, RELEASE=HEAD, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, vendor=Red Hat, Inc., name=centos-stream, vcs-type=git, architecture=x86_64, GIT_REPO=https://github.com/ceph/ceph-container.git, distribution-scope=public, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, version=8, io.buildah.version=1.19.8, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, ceph=True) 2024-03-24T11:24:11.934 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:24:11 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: [24/Mar/2024:11:24:11] ENGINE Bus STOPPING 2024-03-24T11:24:11.934 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:24:11 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: [24/Mar/2024:11:24:11] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-03-24T11:24:11.934 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:24:11 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: [24/Mar/2024:11:24:11] ENGINE Bus STOPPED 2024-03-24T11:24:11.935 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:24:11 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: [24/Mar/2024:11:24:11] ENGINE Bus STARTING 2024-03-24T11:24:11.935 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:24:11 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: [24/Mar/2024:11:24:11] ENGINE Serving on http://:::9283 2024-03-24T11:24:11.935 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:24:11 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: [24/Mar/2024:11:24:11] ENGINE Bus STARTED 2024-03-24T11:24:12.226 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:12 smithi062 podman[93432]: 2024-03-24 11:24:12.108538491 +0000 UTC m=+0.635627500 container cleanup 39110a16ab6274f3f5d12019071044375633567274f6ac19b2c1cc44a62ccb2c (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-c, architecture=x86_64, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, vendor=Red Hat, Inc., GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, io.k8s.display-name=CentOS Stream 8, vcs-type=git, io.openshift.tags=base centos centos-stream, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, distribution-scope=public, release=754, version=8, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., build-date=2022-05-03T08:36:31.336870, com.redhat.component=centos-stream-container, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-17.2.0, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True, RELEASE=HEAD, io.buildah.version=1.19.8, name=centos-stream, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, io.openshift.expose-services=, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com) 2024-03-24T11:24:12.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:12 smithi062 bash[93432]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-c 2024-03-24T11:24:12.724 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:12 smithi062 podman[93457]: 2024-03-24 11:24:12.393531953 +0000 UTC m=+0.742618140 container remove 39110a16ab6274f3f5d12019071044375633567274f6ac19b2c1cc44a62ccb2c (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-c, io.buildah.version=1.19.8, GIT_BRANCH=HEAD, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, GIT_CLEAN=True, io.openshift.expose-services=, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, GIT_REPO=https://github.com/ceph/ceph-container.git, architecture=x86_64, maintainer=Guillaume Abrioux , vendor=Red Hat, Inc., version=8, com.redhat.component=centos-stream-container, build-date=2022-05-03T08:36:31.336870, RELEASE=HEAD, vcs-type=git, CEPH_POINT_RELEASE=-17.2.0, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, distribution-scope=public, name=centos-stream, release=754, io.openshift.tags=base centos centos-stream, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, ceph=True, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, io.k8s.display-name=CentOS Stream 8) 2024-03-24T11:24:12.725 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:12 smithi062 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mon.c.service: Deactivated successfully. 2024-03-24T11:24:12.725 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:12 smithi062 systemd[1]: Stopped Ceph mon.c for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:24:12.725 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:12 smithi062 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mon.c.service: Consumed 11.354s CPU time. 2024-03-24T11:24:12.725 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:12 smithi062 systemd[1]: Starting Ceph mon.c for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:24:12.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:12 smithi062 podman[93590]: 2024-03-24 11:24:12.841399047 +0000 UTC m=+0.018886458 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:24:12.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:12 smithi062 podman[93590]: 2024-03-24 11:24:12.94452977 +0000 UTC m=+0.122017183 container create fe870b225563ceb06347bd74a26a9136b9dd8f6b398ae0beda5ca69a0149bdce (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-c, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-24T11:24:13.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 podman[93590]: 2024-03-24 11:24:13.197559417 +0000 UTC m=+0.375046851 container init fe870b225563ceb06347bd74a26a9136b9dd8f6b398ae0beda5ca69a0149bdce (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-c, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-03-24T11:24:13.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 podman[93590]: 2024-03-24 11:24:13.200509703 +0000 UTC m=+0.377997114 container start fe870b225563ceb06347bd74a26a9136b9dd8f6b398ae0beda5ca69a0149bdce (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-db0330b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, ceph=True) 2024-03-24T11:24:13.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: set uid:gid to 167:167 (ceph:ceph) 2024-03-24T11:24:13.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev), process ceph-mon, pid 2 2024-03-24T11:24:13.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: pidfile_write: ignore empty --pid-file 2024-03-24T11:24:13.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: load: jerasure load: lrc 2024-03-24T11:24:13.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: RocksDB version: 7.9.2 2024-03-24T11:24:13.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Git sha 0 2024-03-24T11:24:13.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Compile date 2024-03-16 10:36:11 2024-03-24T11:24:13.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: DB SUMMARY 2024-03-24T11:24:13.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: DB Session ID: 3OYEAIOTDY2V31109QH5 2024-03-24T11:24:13.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: CURRENT file: CURRENT 2024-03-24T11:24:13.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: IDENTITY file: IDENTITY 2024-03-24T11:24:13.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: MANIFEST file: MANIFEST-000009 size: 2038 Bytes 2024-03-24T11:24:13.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 1, files: 000042.sst 2024-03-24T11:24:13.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000040.log size: 925845 ; 2024-03-24T11:24:13.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.error_if_exists: 0 2024-03-24T11:24:13.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.create_if_missing: 0 2024-03-24T11:24:13.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.paranoid_checks: 1 2024-03-24T11:24:13.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-24T11:24:13.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-24T11:24:13.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-24T11:24:13.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.env: 0x560904a9cae0 2024-03-24T11:24:13.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.fs: PosixFileSystem 2024-03-24T11:24:13.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.info_log: 0x56090653f8c0 2024-03-24T11:24:13.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.max_file_opening_threads: 16 2024-03-24T11:24:13.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.statistics: (nil) 2024-03-24T11:24:13.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.use_fsync: 0 2024-03-24T11:24:13.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.max_log_file_size: 0 2024-03-24T11:24:13.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-24T11:24:13.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-24T11:24:13.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.keep_log_file_num: 1000 2024-03-24T11:24:13.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.recycle_log_file_num: 0 2024-03-24T11:24:13.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.allow_fallocate: 1 2024-03-24T11:24:13.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.allow_mmap_reads: 0 2024-03-24T11:24:13.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.allow_mmap_writes: 0 2024-03-24T11:24:13.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.use_direct_reads: 0 2024-03-24T11:24:13.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-24T11:24:13.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.create_missing_column_families: 0 2024-03-24T11:24:13.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.db_log_dir: 2024-03-24T11:24:13.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.wal_dir: 2024-03-24T11:24:13.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-24T11:24:13.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-24T11:24:13.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-24T11:24:13.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-24T11:24:13.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-24T11:24:13.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-24T11:24:13.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.advise_random_on_open: 1 2024-03-24T11:24:13.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.db_write_buffer_size: 0 2024-03-24T11:24:13.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.write_buffer_manager: 0x560906512dc0 2024-03-24T11:24:13.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-24T11:24:13.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-24T11:24:13.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-24T11:24:13.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.rate_limiter: (nil) 2024-03-24T11:24:13.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-24T11:24:13.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.wal_recovery_mode: 2 2024-03-24T11:24:13.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.enable_thread_tracking: 0 2024-03-24T11:24:13.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.enable_pipelined_write: 0 2024-03-24T11:24:13.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.unordered_write: 0 2024-03-24T11:24:13.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-24T11:24:13.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-24T11:24:13.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-24T11:24:13.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-24T11:24:13.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.row_cache: None 2024-03-24T11:24:13.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.wal_filter: None 2024-03-24T11:24:13.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-24T11:24:13.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.allow_ingest_behind: 0 2024-03-24T11:24:13.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.two_write_queues: 0 2024-03-24T11:24:13.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.manual_wal_flush: 0 2024-03-24T11:24:13.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.wal_compression: 0 2024-03-24T11:24:13.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.atomic_flush: 0 2024-03-24T11:24:13.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-24T11:24:13.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-24T11:24:13.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-24T11:24:13.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.log_readahead_size: 0 2024-03-24T11:24:13.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-24T11:24:13.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.best_efforts_recovery: 0 2024-03-24T11:24:13.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-24T11:24:13.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-24T11:24:13.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.allow_data_in_errors: 0 2024-03-24T11:24:13.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.db_host_id: __hostname__ 2024-03-24T11:24:13.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-24T11:24:13.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.max_background_jobs: 2 2024-03-24T11:24:13.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.max_background_compactions: -1 2024-03-24T11:24:13.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.max_subcompactions: 1 2024-03-24T11:24:13.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-24T11:24:13.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-24T11:24:13.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-24T11:24:13.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.max_total_wal_size: 0 2024-03-24T11:24:13.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-24T11:24:13.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-24T11:24:13.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-24T11:24:13.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-24T11:24:13.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.max_open_files: -1 2024-03-24T11:24:13.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.bytes_per_sync: 0 2024-03-24T11:24:13.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-24T11:24:13.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-24T11:24:13.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.compaction_readahead_size: 0 2024-03-24T11:24:13.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.max_background_flushes: -1 2024-03-24T11:24:13.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Compression algorithms supported: 2024-03-24T11:24:13.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-24T11:24:13.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: kZSTD supported: 0 2024-03-24T11:24:13.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: kXpressCompression supported: 0 2024-03-24T11:24:13.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: kLZ4HCCompression supported: 1 2024-03-24T11:24:13.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: kZlibCompression supported: 1 2024-03-24T11:24:13.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: kSnappyCompression supported: 1 2024-03-24T11:24:13.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: kLZ4Compression supported: 1 2024-03-24T11:24:13.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: kBZip2Compression supported: 0 2024-03-24T11:24:13.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-24T11:24:13.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-24T11:24:13.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000009 2024-03-24T11:24:13.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-24T11:24:13.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-24T11:24:13.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.merge_operator: 2024-03-24T11:24:13.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.compaction_filter: None 2024-03-24T11:24:13.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.compaction_filter_factory: None 2024-03-24T11:24:13.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.sst_partitioner_factory: None 2024-03-24T11:24:13.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-24T11:24:13.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-24T11:24:13.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56090653f9c0) 2024-03-24T11:24:13.487 INFO:journalctl@ceph.mon.c.smithi062.stdout: cache_index_and_filter_blocks: 1 2024-03-24T11:24:13.487 INFO:journalctl@ceph.mon.c.smithi062.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-24T11:24:13.487 INFO:journalctl@ceph.mon.c.smithi062.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-24T11:24:13.487 INFO:journalctl@ceph.mon.c.smithi062.stdout: pin_top_level_index_and_filter: 1 2024-03-24T11:24:13.487 INFO:journalctl@ceph.mon.c.smithi062.stdout: index_type: 0 2024-03-24T11:24:13.487 INFO:journalctl@ceph.mon.c.smithi062.stdout: data_block_index_type: 0 2024-03-24T11:24:13.487 INFO:journalctl@ceph.mon.c.smithi062.stdout: index_shortening: 1 2024-03-24T11:24:13.487 INFO:journalctl@ceph.mon.c.smithi062.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-24T11:24:13.487 INFO:journalctl@ceph.mon.c.smithi062.stdout: checksum: 4 2024-03-24T11:24:13.487 INFO:journalctl@ceph.mon.c.smithi062.stdout: no_block_cache: 0 2024-03-24T11:24:13.488 INFO:journalctl@ceph.mon.c.smithi062.stdout: block_cache: 0x560906543090 2024-03-24T11:24:13.488 INFO:journalctl@ceph.mon.c.smithi062.stdout: block_cache_name: BinnedLRUCache 2024-03-24T11:24:13.488 INFO:journalctl@ceph.mon.c.smithi062.stdout: block_cache_options: 2024-03-24T11:24:13.488 INFO:journalctl@ceph.mon.c.smithi062.stdout: capacity : 536870912 2024-03-24T11:24:13.488 INFO:journalctl@ceph.mon.c.smithi062.stdout: num_shard_bits : 4 2024-03-24T11:24:13.488 INFO:journalctl@ceph.mon.c.smithi062.stdout: strict_capacity_limit : 0 2024-03-24T11:24:13.488 INFO:journalctl@ceph.mon.c.smithi062.stdout: high_pri_pool_ratio: 0.000 2024-03-24T11:24:13.488 INFO:journalctl@ceph.mon.c.smithi062.stdout: block_cache_compressed: (nil) 2024-03-24T11:24:13.488 INFO:journalctl@ceph.mon.c.smithi062.stdout: persistent_cache: (nil) 2024-03-24T11:24:13.488 INFO:journalctl@ceph.mon.c.smithi062.stdout: block_size: 4096 2024-03-24T11:24:13.488 INFO:journalctl@ceph.mon.c.smithi062.stdout: block_size_deviation: 10 2024-03-24T11:24:13.488 INFO:journalctl@ceph.mon.c.smithi062.stdout: block_restart_interval: 16 2024-03-24T11:24:13.488 INFO:journalctl@ceph.mon.c.smithi062.stdout: index_block_restart_interval: 1 2024-03-24T11:24:13.489 INFO:journalctl@ceph.mon.c.smithi062.stdout: metadata_block_size: 4096 2024-03-24T11:24:13.489 INFO:journalctl@ceph.mon.c.smithi062.stdout: partition_filters: 0 2024-03-24T11:24:13.489 INFO:journalctl@ceph.mon.c.smithi062.stdout: use_delta_encoding: 1 2024-03-24T11:24:13.489 INFO:journalctl@ceph.mon.c.smithi062.stdout: filter_policy: bloomfilter 2024-03-24T11:24:13.489 INFO:journalctl@ceph.mon.c.smithi062.stdout: whole_key_filtering: 1 2024-03-24T11:24:13.489 INFO:journalctl@ceph.mon.c.smithi062.stdout: verify_compression: 0 2024-03-24T11:24:13.489 INFO:journalctl@ceph.mon.c.smithi062.stdout: read_amp_bytes_per_bit: 0 2024-03-24T11:24:13.489 INFO:journalctl@ceph.mon.c.smithi062.stdout: format_version: 5 2024-03-24T11:24:13.489 INFO:journalctl@ceph.mon.c.smithi062.stdout: enable_index_compression: 1 2024-03-24T11:24:13.489 INFO:journalctl@ceph.mon.c.smithi062.stdout: block_align: 0 2024-03-24T11:24:13.489 INFO:journalctl@ceph.mon.c.smithi062.stdout: max_auto_readahead_size: 262144 2024-03-24T11:24:13.489 INFO:journalctl@ceph.mon.c.smithi062.stdout: prepopulate_block_cache: 0 2024-03-24T11:24:13.489 INFO:journalctl@ceph.mon.c.smithi062.stdout: initial_auto_readahead_size: 8192 2024-03-24T11:24:13.489 INFO:journalctl@ceph.mon.c.smithi062.stdout: num_file_reads_for_auto_readahead: 2 2024-03-24T11:24:13.490 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.write_buffer_size: 33554432 2024-03-24T11:24:13.490 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.max_write_buffer_number: 2 2024-03-24T11:24:13.490 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.compression: NoCompression 2024-03-24T11:24:13.490 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.bottommost_compression: Disabled 2024-03-24T11:24:13.490 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.prefix_extractor: nullptr 2024-03-24T11:24:13.490 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-24T11:24:13.490 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.num_levels: 7 2024-03-24T11:24:13.490 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-24T11:24:13.490 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-24T11:24:13.490 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-24T11:24:13.490 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-24T11:24:13.490 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-24T11:24:13.490 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-24T11:24:13.490 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-24T11:24:13.491 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-24T11:24:13.491 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-24T11:24:13.491 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-24T11:24:13.491 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-24T11:24:13.491 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-24T11:24:13.491 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-24T11:24:13.491 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.compression_opts.level: 32767 2024-03-24T11:24:13.491 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.compression_opts.strategy: 0 2024-03-24T11:24:13.491 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-24T11:24:13.491 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-24T11:24:13.491 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-24T11:24:13.491 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-24T11:24:13.491 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.compression_opts.enabled: false 2024-03-24T11:24:13.491 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-24T11:24:13.492 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-24T11:24:13.492 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-24T11:24:13.492 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-24T11:24:13.492 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.target_file_size_base: 67108864 2024-03-24T11:24:13.492 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-24T11:24:13.492 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-24T11:24:13.492 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-24T11:24:13.492 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-24T11:24:13.492 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-24T11:24:13.492 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-24T11:24:13.492 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-24T11:24:13.492 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-24T11:24:13.492 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-24T11:24:13.492 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-24T11:24:13.493 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-24T11:24:13.493 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-24T11:24:13.493 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-24T11:24:13.493 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-24T11:24:13.493 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.arena_block_size: 1048576 2024-03-24T11:24:13.493 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-24T11:24:13.493 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-24T11:24:13.493 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.disable_auto_compactions: 0 2024-03-24T11:24:13.493 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-24T11:24:13.493 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-24T11:24:13.493 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-24T11:24:13.493 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-24T11:24:13.494 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-24T11:24:13.494 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-24T11:24:13.494 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-24T11:24:13.494 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-24T11:24:13.494 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-24T11:24:13.494 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-24T11:24:13.494 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.table_properties_collectors: 2024-03-24T11:24:13.494 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.inplace_update_support: 0 2024-03-24T11:24:13.494 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-24T11:24:13.494 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-24T11:24:13.495 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-24T11:24:13.495 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-24T11:24:13.495 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.bloom_locality: 0 2024-03-24T11:24:13.495 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.max_successive_merges: 0 2024-03-24T11:24:13.495 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-24T11:24:13.495 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.paranoid_file_checks: 0 2024-03-24T11:24:13.495 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.force_consistency_checks: 1 2024-03-24T11:24:13.495 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.report_bg_io_stats: 0 2024-03-24T11:24:13.495 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.ttl: 2592000 2024-03-24T11:24:13.495 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-24T11:24:13.495 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-24T11:24:13.495 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-24T11:24:13.495 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.enable_blob_files: false 2024-03-24T11:24:13.495 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.min_blob_size: 0 2024-03-24T11:24:13.496 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.blob_file_size: 268435456 2024-03-24T11:24:13.496 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-24T11:24:13.496 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-24T11:24:13.496 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-24T11:24:13.496 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-24T11:24:13.496 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-24T11:24:13.496 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.blob_file_starting_level: 0 2024-03-24T11:24:13.496 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-24T11:24:13.496 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 42.sst 2024-03-24T11:24:13.496 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-24T11:24:13.496 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 44, last_sequence is 25803, log_number is 40,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-24T11:24:13.496 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 40 2024-03-24T11:24:13.496 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: b6db2122-ff52-4d85-81c5-579d4a8837d2 2024-03-24T11:24:13.497 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711279453229329, "job": 1, "event": "recovery_started", "wal_files": [40]} 2024-03-24T11:24:13.497 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #40 mode 2 2024-03-24T11:24:13.497 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711279453232727, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 45, "file_size": 589259, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 25808, "largest_seqno": 26570, "table_properties": {"data_size": 585798, "index_size": 1359, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 1093, "raw_key_size": 10444, "raw_average_key_size": 24, "raw_value_size": 578410, "raw_average_value_size": 1370, "num_data_blocks": 55, "num_entries": 422, "num_filter_entries": 422, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711279453, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "b6db2122-ff52-4d85-81c5-579d4a8837d2", "db_session_id": "3OYEAIOTDY2V31109QH5", "orig_file_number": 45, "seqno_to_time_mapping": "N/A"}} 2024-03-24T11:24:13.497 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711279453232820, "job": 1, "event": "recovery_finished"} 2024-03-24T11:24:13.497 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: [db/version_set.cc:5047] Creating manifest 47 2024-03-24T11:24:13.497 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-24T11:24:13.497 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000040.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-24T11:24:13.497 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x560906634000 2024-03-24T11:24:13.497 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: DB pointer 0x56090661e000 2024-03-24T11:24:13.497 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-24T11:24:13.497 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-24T11:24:13.497 INFO:journalctl@ceph.mon.c.smithi062.stdout: ** DB Stats ** 2024-03-24T11:24:13.497 INFO:journalctl@ceph.mon.c.smithi062.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-24T11:24:13.497 INFO:journalctl@ceph.mon.c.smithi062.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-24T11:24:13.497 INFO:journalctl@ceph.mon.c.smithi062.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-24T11:24:13.498 INFO:journalctl@ceph.mon.c.smithi062.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-24T11:24:13.498 INFO:journalctl@ceph.mon.c.smithi062.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-24T11:24:13.498 INFO:journalctl@ceph.mon.c.smithi062.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-24T11:24:13.498 INFO:journalctl@ceph.mon.c.smithi062.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-24T11:24:13.498 INFO:journalctl@ceph.mon.c.smithi062.stdout: 2024-03-24T11:24:13.498 INFO:journalctl@ceph.mon.c.smithi062.stdout: ** Compaction Stats [default] ** 2024-03-24T11:24:13.498 INFO:journalctl@ceph.mon.c.smithi062.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-24T11:24:13.498 INFO:journalctl@ceph.mon.c.smithi062.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-24T11:24:13.498 INFO:journalctl@ceph.mon.c.smithi062.stdout: L0 1/0 575.45 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 308.3 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-03-24T11:24:13.498 INFO:journalctl@ceph.mon.c.smithi062.stdout: L6 1/0 11.02 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-03-24T11:24:13.498 INFO:journalctl@ceph.mon.c.smithi062.stdout: Sum 2/0 11.59 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 308.3 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-03-24T11:24:13.498 INFO:journalctl@ceph.mon.c.smithi062.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 308.3 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-03-24T11:24:13.498 INFO:journalctl@ceph.mon.c.smithi062.stdout: 2024-03-24T11:24:13.499 INFO:journalctl@ceph.mon.c.smithi062.stdout: ** Compaction Stats [default] ** 2024-03-24T11:24:13.499 INFO:journalctl@ceph.mon.c.smithi062.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-24T11:24:13.499 INFO:journalctl@ceph.mon.c.smithi062.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-24T11:24:13.499 INFO:journalctl@ceph.mon.c.smithi062.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 308.3 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-03-24T11:24:13.499 INFO:journalctl@ceph.mon.c.smithi062.stdout: 2024-03-24T11:24:13.499 INFO:journalctl@ceph.mon.c.smithi062.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-24T11:24:13.499 INFO:journalctl@ceph.mon.c.smithi062.stdout: 2024-03-24T11:24:13.499 INFO:journalctl@ceph.mon.c.smithi062.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-24T11:24:13.499 INFO:journalctl@ceph.mon.c.smithi062.stdout: Flush(GB): cumulative 0.001, interval 0.001 2024-03-24T11:24:13.499 INFO:journalctl@ceph.mon.c.smithi062.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-24T11:24:13.499 INFO:journalctl@ceph.mon.c.smithi062.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-24T11:24:13.499 INFO:journalctl@ceph.mon.c.smithi062.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-24T11:24:13.499 INFO:journalctl@ceph.mon.c.smithi062.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-24T11:24:13.499 INFO:journalctl@ceph.mon.c.smithi062.stdout: Cumulative compaction: 0.00 GB write, 72.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-24T11:24:13.500 INFO:journalctl@ceph.mon.c.smithi062.stdout: Interval compaction: 0.00 GB write, 72.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-24T11:24:13.500 INFO:journalctl@ceph.mon.c.smithi062.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-24T11:24:13.500 INFO:journalctl@ceph.mon.c.smithi062.stdout: Block cache BinnedLRUCache@0x560906543090#2 capacity: 512.00 MB usage: 2.61 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-03-24T11:24:13.500 INFO:journalctl@ceph.mon.c.smithi062.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,1.16 KB,0.000220537%) IndexBlock(1,1.45 KB,0.000277162%) Misc(1,0.00 KB,0%) 2024-03-24T11:24:13.500 INFO:journalctl@ceph.mon.c.smithi062.stdout: 2024-03-24T11:24:13.500 INFO:journalctl@ceph.mon.c.smithi062.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-24T11:24:13.500 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: starting mon.c rank 1 at public addrs [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] at bind addrs [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] mon_data /var/lib/ceph/mon/ceph-c fsid ebb89568-e9ce-11ee-95cd-87774f69a715 2024-03-24T11:24:13.500 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: mon.c@-1(???) e3 preinit fsid ebb89568-e9ce-11ee-95cd-87774f69a715 2024-03-24T11:24:13.500 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: mon.c@-1(???).mds e1 new map 2024-03-24T11:24:13.500 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: mon.c@-1(???).mds e1 print_map 2024-03-24T11:24:13.500 INFO:journalctl@ceph.mon.c.smithi062.stdout: e1 2024-03-24T11:24:13.500 INFO:journalctl@ceph.mon.c.smithi062.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T11:24:13.500 INFO:journalctl@ceph.mon.c.smithi062.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T11:24:13.501 INFO:journalctl@ceph.mon.c.smithi062.stdout: legacy client fscid: -1 2024-03-24T11:24:13.501 INFO:journalctl@ceph.mon.c.smithi062.stdout: 2024-03-24T11:24:13.501 INFO:journalctl@ceph.mon.c.smithi062.stdout: No filesystems configured 2024-03-24T11:24:13.501 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: mon.c@-1(???).osd e108 crush map has features 3314933000854323200, adjusting msgr requires 2024-03-24T11:24:13.501 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: mon.c@-1(???).osd e108 crush map has features 432629239337189376, adjusting msgr requires 2024-03-24T11:24:13.501 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: mon.c@-1(???).osd e108 crush map has features 432629239337189376, adjusting msgr requires 2024-03-24T11:24:13.501 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: mon.c@-1(???).osd e108 crush map has features 432629239337189376, adjusting msgr requires 2024-03-24T11:24:13.501 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 ceph-mon[93609]: mon.c@-1(???).paxosservice(auth 1..26) refresh upgraded, format 0 -> 3 2024-03-24T11:24:13.501 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 bash[93590]: fe870b225563ceb06347bd74a26a9136b9dd8f6b398ae0beda5ca69a0149bdce 2024-03-24T11:24:13.501 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:13 smithi062 systemd[1]: Started Ceph mon.c for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:24:14.538 INFO:teuthology.orchestra.run.smithi062.stdout:true 2024-03-24T11:24:15.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:14 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:15.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:14 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:15.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:14 smithi139 ceph-mon[25719]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:15.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:14 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:24:15.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:14 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T11:24:15.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:14 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T11:24:15.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:14 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:24:15.223 INFO:teuthology.orchestra.run.smithi062.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T11:24:15.223 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi062 smithi062 running 8s ago 13m - - 2024-03-24T11:24:15.223 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi139 smithi139 running 4s ago 13m - - 2024-03-24T11:24:15.223 INFO:teuthology.orchestra.run.smithi062.stdout:alertmanager.a smithi062 *:9093,9094 running (3m) 8s ago 8m 25.1M - 0.23.0 ba2b418f427c 818298d58140 2024-03-24T11:24:15.223 INFO:teuthology.orchestra.run.smithi062.stdout:grafana.a smithi139 *:3000 running (2m) 4s ago 8m 51.7M - 8.3.5 dad864ee21e9 b48167e7171e 2024-03-24T11:24:15.223 INFO:teuthology.orchestra.run.smithi062.stdout:iscsi.foo.smithi062.dggsvk smithi062 running (108s) 8s ago 7m 53.3M - 3.5 e1d6a67b021e 0a98806ec839 2024-03-24T11:24:15.223 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.x smithi139 *:8443,9283,8765 running (60s) 4s ago 12m 447M - 19.0.0-1667-gdb0330b1 4f325f8440e9 d94987b11387 2024-03-24T11:24:15.223 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.y smithi062 *:8443,9283,8765 running (2m) 8s ago 14m 501M - 19.0.0-1667-gdb0330b1 4f325f8440e9 ca1a49028d65 2024-03-24T11:24:15.223 INFO:teuthology.orchestra.run.smithi062.stdout:mon.a smithi062 running (16s) 8s ago 14m 37.7M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 f7afe6e28c88 2024-03-24T11:24:15.223 INFO:teuthology.orchestra.run.smithi062.stdout:mon.b smithi139 running (13m) 4s ago 13m 59.0M 2048M 17.2.0 e1d6a67b021e 8945cdb4e6aa 2024-03-24T11:24:15.223 INFO:teuthology.orchestra.run.smithi062.stdout:mon.c smithi062 starting - - - 2048M 2024-03-24T11:24:15.224 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.a smithi062 *:9100 running (3m) 8s ago 9m 15.8M - 1.3.1 1dbe0e931976 79d5a02a999f 2024-03-24T11:24:15.224 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.b smithi139 *:9100 running (3m) 4s ago 9m 15.8M - 1.3.1 1dbe0e931976 8f9f4608e465 2024-03-24T11:24:15.224 INFO:teuthology.orchestra.run.smithi062.stdout:osd.0 smithi062 running (12m) 8s ago 12m 53.6M 4096M 17.2.0 e1d6a67b021e 8040b30d7170 2024-03-24T11:24:15.224 INFO:teuthology.orchestra.run.smithi062.stdout:osd.1 smithi062 running (12m) 8s ago 12m 55.4M 4096M 17.2.0 e1d6a67b021e e62d22e5dc29 2024-03-24T11:24:15.224 INFO:teuthology.orchestra.run.smithi062.stdout:osd.2 smithi062 running (11m) 8s ago 11m 48.7M 4096M 17.2.0 e1d6a67b021e 714f550103a7 2024-03-24T11:24:15.224 INFO:teuthology.orchestra.run.smithi062.stdout:osd.3 smithi062 running (11m) 8s ago 11m 53.2M 4096M 17.2.0 e1d6a67b021e 1d2868bd5195 2024-03-24T11:24:15.224 INFO:teuthology.orchestra.run.smithi062.stdout:osd.4 smithi139 running (11m) 4s ago 11m 57.7M 4096M 17.2.0 e1d6a67b021e 7568bfa9e74d 2024-03-24T11:24:15.224 INFO:teuthology.orchestra.run.smithi062.stdout:osd.5 smithi139 running (10m) 4s ago 10m 50.9M 4096M 17.2.0 e1d6a67b021e dd0abd599a14 2024-03-24T11:24:15.224 INFO:teuthology.orchestra.run.smithi062.stdout:osd.6 smithi139 running (10m) 4s ago 10m 50.5M 4096M 17.2.0 e1d6a67b021e a4b1289aca69 2024-03-24T11:24:15.224 INFO:teuthology.orchestra.run.smithi062.stdout:osd.7 smithi139 running (9m) 4s ago 9m 53.1M 4096M 17.2.0 e1d6a67b021e bc7fadbc1384 2024-03-24T11:24:15.224 INFO:teuthology.orchestra.run.smithi062.stdout:prometheus.a smithi139 *:9095 running (2m) 4s ago 9m 60.0M - 2.33.4 514e6a882f6e 2ed47f810d33 2024-03-24T11:24:15.224 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi062.sacpds smithi062 *:8000 running (7m) 8s ago 7m 72.8M - 17.2.0 e1d6a67b021e 89f4ac308e98 2024-03-24T11:24:15.224 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi139.yljesh smithi139 *:8000 running (7m) 4s ago 7m 71.6M - 17.2.0 e1d6a67b021e e189bdc12cae 2024-03-24T11:24:15.224 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi062.xvpxnk smithi062 *:80 running (7m) 8s ago 7m 72.6M - 17.2.0 e1d6a67b021e d1aa74adab44 2024-03-24T11:24:15.224 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi139.pbtrmp smithi139 *:80 running (7m) 4s ago 7m 75.1M - 17.2.0 e1d6a67b021e 7f8e336755d7 2024-03-24T11:24:15.225 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:14 smithi062 ceph-mon[93609]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:15.225 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:14 smithi062 ceph-mon[93609]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:15.225 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:14 smithi062 ceph-mon[93609]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:15.225 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:14 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:24:15.225 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:14 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T11:24:15.225 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:14 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T11:24:15.225 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:14 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:24:15.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:14 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:15.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:14 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:15.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:14 smithi062 ceph-mon[90553]: from='mgr.25489 ' entity='mgr.y' 2024-03-24T11:24:15.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:14 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:24:15.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:14 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T11:24:15.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:14 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T11:24:15.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:14 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:24:15.649 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:24:15.649 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": { 2024-03-24T11:24:15.649 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 1, 2024-03-24T11:24:15.649 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-24T11:24:15.649 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:24:15.650 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": { 2024-03-24T11:24:15.650 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-24T11:24:15.650 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:24:15.650 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": { 2024-03-24T11:24:15.650 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 8 2024-03-24T11:24:15.650 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:24:15.650 INFO:teuthology.orchestra.run.smithi062.stdout: "rgw": { 2024-03-24T11:24:15.650 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4 2024-03-24T11:24:15.650 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:24:15.650 INFO:teuthology.orchestra.run.smithi062.stdout: "overall": { 2024-03-24T11:24:15.650 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 13, 2024-03-24T11:24:15.650 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 4 2024-03-24T11:24:15.650 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-24T11:24:15.650 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:24:15.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:24:15.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: pgmap v19: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:24:15.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: mon.c calling monitor election 2024-03-24T11:24:15.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: from='client.44100 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:24:15.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: mon.a calling monitor election 2024-03-24T11:24:15.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-24T11:24:15.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: monmap e3: 3 mons at {a=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0],b=[v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0],c=[v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-24T11:24:15.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: fsmap 2024-03-24T11:24:15.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: osdmap e108: 8 total, 8 up, 8 in 2024-03-24T11:24:15.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: mgrmap e44: y(active, since 33s), standbys: x 2024-03-24T11:24:15.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: overall HEALTH_OK 2024-03-24T11:24:15.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:15.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:15.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:15.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:15.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:15.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:15.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: from='client.44106 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:24:15.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:15.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:15.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:15.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:15.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:15.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:15.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:15.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:15.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:15.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:15.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:15.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:15.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:15.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:24:15.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: pgmap v19: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:24:15.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: mon.c calling monitor election 2024-03-24T11:24:15.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: from='client.44100 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:24:15.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: mon.a calling monitor election 2024-03-24T11:24:15.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-24T11:24:15.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: monmap e3: 3 mons at {a=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0],b=[v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0],c=[v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-24T11:24:15.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: fsmap 2024-03-24T11:24:15.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: osdmap e108: 8 total, 8 up, 8 in 2024-03-24T11:24:15.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: mgrmap e44: y(active, since 33s), standbys: x 2024-03-24T11:24:15.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: overall HEALTH_OK 2024-03-24T11:24:15.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:15.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:15.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:15.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:15.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:15.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:15.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: from='client.44106 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:24:15.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:15.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:15.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:15.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:15.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:15.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:15.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:15.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:15.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:15.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:15.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:15.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:15.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:15.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:15.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:15.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:15.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:15.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:15.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:15.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:15.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:15.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:15.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:15.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:15.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:15.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:15.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:15.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:15.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:15.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:15.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:15.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: from='client.? 172.21.15.62:0/2750795407' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:15.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:15.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:15.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:15.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:15.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:15.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:15.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:15.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:15.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:15.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:15.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:15.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:15.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:15.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:15.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:15.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:15.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:15.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:15.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:15.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:15.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:15.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:15.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: from='client.? 172.21.15.62:0/2750795407' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:15.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:15.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:15.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:15.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:15 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:16.003 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:24:16.003 INFO:teuthology.orchestra.run.smithi062.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69", 2024-03-24T11:24:16.003 INFO:teuthology.orchestra.run.smithi062.stdout: "in_progress": true, 2024-03-24T11:24:16.003 INFO:teuthology.orchestra.run.smithi062.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T11:24:16.003 INFO:teuthology.orchestra.run.smithi062.stdout: "services_complete": [ 2024-03-24T11:24:16.003 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr" 2024-03-24T11:24:16.003 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-03-24T11:24:16.003 INFO:teuthology.orchestra.run.smithi062.stdout: "progress": "3/23 daemons upgraded", 2024-03-24T11:24:16.003 INFO:teuthology.orchestra.run.smithi062.stdout: "message": "Currently upgrading mon daemons", 2024-03-24T11:24:16.003 INFO:teuthology.orchestra.run.smithi062.stdout: "is_paused": false 2024-03-24T11:24:16.004 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:24:16.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:24:16.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: pgmap v19: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:24:16.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: mon.c calling monitor election 2024-03-24T11:24:16.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: from='client.44100 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:24:16.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: mon.a calling monitor election 2024-03-24T11:24:16.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-24T11:24:16.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: monmap e3: 3 mons at {a=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0],b=[v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0],c=[v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-24T11:24:16.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: fsmap 2024-03-24T11:24:16.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: osdmap e108: 8 total, 8 up, 8 in 2024-03-24T11:24:16.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: mgrmap e44: y(active, since 33s), standbys: x 2024-03-24T11:24:16.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: overall HEALTH_OK 2024-03-24T11:24:16.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:16.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:16.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:16.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:16.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:16.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:16.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: from='client.44106 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:24:16.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:16.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:16.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:16.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:16.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:16.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:16.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:16.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:16.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:16.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:16.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:16.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:16.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:16.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:16.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:16.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:16.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:16.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:16.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:16.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:16.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:16.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:16.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:16.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:16.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:16.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:16.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:16.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:16.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:16.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:16.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:16.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/2750795407' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:16.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:16.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:16.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:16.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:15 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:16.470 INFO:teuthology.orchestra.run.smithi062.stdout:HEALTH_OK 2024-03-24T11:24:16.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[90553]: pgmap v20: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:24:16.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:16.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[90553]: from='client.44112 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:24:16.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:16.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:16.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:16.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:16.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:16.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:16.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:16.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:16.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:16.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:16.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:16.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:16.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:16.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:16.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:16.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:16.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:16.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:16.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:16.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:16.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:16.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:16.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:16.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:16.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:16.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:16.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:16.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[93609]: pgmap v20: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:24:16.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:16.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[93609]: from='client.44112 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:24:16.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:16.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:16.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:16.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:16.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:16.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:16.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:16.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:16.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:16.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:16.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:16.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:16.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:16.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:16.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:16.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:16.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:16.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:16.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:16.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:16.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:16.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:16.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:16.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:16.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:16.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:16.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:16.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:16.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:16.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:16.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:16.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:16.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[93609]: from='client.? 172.21.15.62:0/1289980576' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:24:16.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:16.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:16.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:16.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:16.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:16.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:16.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:16.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:16.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:16.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:16.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:16.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:16.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:16.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:16.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:16.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:16.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:16.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:16.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:16.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:16.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[90553]: from='client.? 172.21.15.62:0/1289980576' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:24:16.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:16.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:16.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:16.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:16.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:16.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:16.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:16.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:16.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:16.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:16.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:16.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:16.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:16.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:16.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:17.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:16 smithi139 ceph-mon[25719]: pgmap v20: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:24:17.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:16 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:17.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:16 smithi139 ceph-mon[25719]: from='client.44112 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:24:17.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:16 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:17.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:16 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:17.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:16 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:17.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:16 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:17.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:16 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:17.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:16 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:17.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:16 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:17.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:16 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:17.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:16 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:17.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:16 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:17.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:16 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:17.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:16 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:17.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:16 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:17.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:16 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:17.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:16 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:17.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:16 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:17.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:16 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:17.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:16 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:17.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:16 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:17.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:16 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:17.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:16 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:17.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:16 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:17.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:16 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:17.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:16 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:17.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:16 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:17.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:16 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:17.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:16 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:17.115 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:16 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:17.115 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:16 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:17.115 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:16 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:17.115 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:16 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:17.115 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:16 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:17.115 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:16 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/1289980576' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:24:17.115 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:16 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:17.116 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:16 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:17.116 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:16 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:17.116 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:16 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:17.116 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:16 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:17.116 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:16 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:17.116 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:16 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:17.117 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:16 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:17.117 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:16 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:17.117 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:16 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:17.117 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:16 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:17.117 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:16 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:17.117 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:16 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:17.118 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:16 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:17.118 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:16 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:18.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='client.44124 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:24:18.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:18.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:18.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:18.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:18.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:18.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:18.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:18.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:18.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:18.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:18.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:18.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:18.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:18.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:18.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:18.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:18.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:18.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:18.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:18.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:18.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:18.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:18.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:18.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:18.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:18.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:18.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:18.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:18.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:18.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:18.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:18.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:18.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:18.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:18.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:18.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:18.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:18.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:18.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:18.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:18.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:18.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:18.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:18.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:18.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:18.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:18.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:18.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:18.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:18.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:18.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:18.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:18.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:18.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:18.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:18.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:18.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:18.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:18.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:18.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:18.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:18.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:18.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:18.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:18.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:17 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:18.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='client.44124 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:24:18.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:18.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:18.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:18.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:18.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:18.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:18.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:18.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:18.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:18.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:18.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:18.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:18.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:18.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:18.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:18.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:18.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:18.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:18.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:18.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:18.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:18.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:18.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:18.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:18.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:18.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:18.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:18.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:18.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:18.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='client.44124 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:24:18.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:18.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:18.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:18.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:18.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:18.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:18.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:18.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:18.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:18.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:18.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:18.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:18.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:18.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:18.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:18.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:18.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:18.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:18.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:18.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:18.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:18.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:18.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:18.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:18.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:18.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:18.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:18.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:18.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:18.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:18.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:18.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:18.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:18.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:18.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:18.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:18.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:18.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:18.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:18.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:18.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:18.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:18.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:18.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:18.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:18.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:18.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:18.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:18.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:18.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:18.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:18.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:18.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:18.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:18.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:18.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:18.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:18.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:18.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:18.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:18.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:18.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:18.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:18.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:18.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:18.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:18.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:18.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:18.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:18.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:18.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:18.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:18.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:18.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:18.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:18.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:18.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:18.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:18.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:18.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:18.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:18.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:18.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:18.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:18.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:18.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:18.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:18.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:18.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:18.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:18.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:18.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:18.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:18.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:18.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:18.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:18.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:18.238 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:18.238 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:18.238 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:18.238 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:18.560 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:24:18.241+0000 7f4e4aad7700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-24T11:24:18.900 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: pgmap v21: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:24:18.900 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:18.900 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:18.900 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:18.900 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:18.900 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:18.900 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:18.901 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:18.901 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:18.901 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:18.901 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:18.901 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:18.901 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:18.901 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:18.901 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:18.901 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:18.901 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:18.901 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:18.901 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:18.901 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:18.902 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:18.902 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:18.902 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:18.902 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:18.902 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:18.902 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:18.902 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:18.902 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:18.902 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:18.902 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:18.902 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: pgmap v21: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:24:18.903 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:18.903 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:18.903 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:18.903 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:18.903 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:18.903 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:18.903 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:18.903 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:18.903 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:18.903 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:18.903 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:18.903 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:18.903 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:18.903 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:18.903 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:18.904 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:18.904 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:18.904 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:18.904 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:18.904 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:18.904 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:18.904 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:18.904 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:18.904 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:18.904 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:18.904 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:18.904 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:18.904 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:18.904 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:18.905 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:18.905 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:18.905 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:18.905 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:18.905 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:18.905 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:18.905 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:18.905 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:18.905 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:18.905 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:18.905 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:18.905 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:18.906 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:18.906 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:18.906 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:18.906 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:18.906 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:18.906 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:18.906 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:18.906 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:18.906 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:18.906 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:18.906 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:18.906 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:18.906 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:18.906 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:18.907 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:18.907 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:18.907 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:18.907 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:18.907 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:18.907 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:18.907 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:18.907 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:18.907 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:18.907 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:18.907 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:18.907 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:18.908 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:18.908 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:18.908 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:18.908 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:18.908 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:18.908 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:18.908 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:18.908 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:18.908 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:18.908 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:18.908 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:18.908 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:18.908 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:18.908 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:18 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:19.108 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:24:18.658Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"smithi062:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"172.21.15.139:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-24T11:24:19.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: pgmap v21: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:24:19.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:19.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:19.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:19.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:19.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:19.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:19.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:19.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:19.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:19.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:19.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:19.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:19.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:19.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:19.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:19.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:19.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:19.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:19.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:19.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:19.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:19.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:19.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:19.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:19.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:19.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:19.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:19.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:19.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:19.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:19.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:19.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:19.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:19.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:19.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:19.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:19.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:19.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:19.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:19.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:19.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:19.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:19.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:19.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:19.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:19.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:19.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:19.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:19.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:19.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:19.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:19.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:19.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:19.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:19.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:18 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:19.951 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:19.951 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:19.951 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:19.952 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:19.952 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:19.952 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:19.952 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:19.952 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:19.952 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:19.952 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:19.952 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:19.952 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:19.952 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:19.952 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:19.952 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:19.952 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:19.952 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:19.953 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:19.953 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:19.953 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:19.953 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:19.953 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:19.953 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:19.953 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:19.953 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:19.953 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:19.953 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:19.953 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:19.953 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:19.953 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:19.954 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:19.954 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:19.954 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:19.954 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:19.954 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:19.954 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:19.954 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:19.954 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:19.954 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:19.954 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:19.954 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:19.954 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:19.954 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:19.955 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:19.955 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:19.955 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:19.955 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:19.955 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:19.955 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:19.955 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:19.955 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:19.955 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:19.955 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:19.955 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:19.955 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:19.955 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:19.955 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:19.956 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:19.956 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:19.956 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:19.956 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:19.956 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:19.956 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:19.956 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:19.956 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:19.956 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:19.956 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:19.956 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:19.957 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:19.957 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:19.957 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:19.957 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:19.957 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:19.957 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:19.957 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:19.957 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:19.957 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:19.957 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:19.957 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:19.957 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:19.957 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:19.957 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:19.957 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:19.958 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:19.958 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:19.958 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:19.958 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:19.958 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:19.958 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:19.958 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:19.958 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:19.958 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:19.958 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:19.958 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:19.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:19.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:19.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:19.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:19.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:19.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:19.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:19.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:19.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:19.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:19.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:19.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:19.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:19.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:19.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:19.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:19.961 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:19.961 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:19.961 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:19.961 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:19.961 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:19.961 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:19.962 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:19.962 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:19.962 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:19.962 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:19.962 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:19.962 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:19.962 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:19.962 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:19.962 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:19.962 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:19.962 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:19.963 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:20.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:20.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:20.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:20.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:20.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:20.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:20.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:20.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:20.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:20.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:20.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:20.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:20.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:20.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:20.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:20.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:20.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:20.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:20.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:20.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:20.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:20.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:20.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:20.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:20.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:20.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:20.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:20.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:20.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:20.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:20.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:20.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:20.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:20.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:20.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:20.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:20.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:20.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:20.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:20.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:20.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:20.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:20.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:20.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:20.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:20.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:20.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:20.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:20.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:20.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:20.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:20.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:20.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:20.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:20.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:20.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:20.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:20.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:20.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:20.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:20.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:20.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:20.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:20.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:19 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:20.976 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:24:20] "GET /metrics HTTP/1.1" 200 38323 "" "Prometheus/2.33.4" 2024-03-24T11:24:20.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:20.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: pgmap v22: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:24:20.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:20.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:20.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:20.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:20.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:20.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:20.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:20.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:20.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:20.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:20.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:20.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:20.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:20.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:20.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:20.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:20.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:20.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:20.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:20.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:20.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:20.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:20.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:20.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:20.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:20.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:20.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:20.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:20.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:20.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:20.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:20.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:20.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:20.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:20.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:20.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:20.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:20.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:20.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:20.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:20.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:20.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:20.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:20.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:20.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:20.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:20.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:20.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:20.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:20.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:20.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:20.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:20.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:20.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:20.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:20.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:20.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: pgmap v22: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:24:20.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:20.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:20.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:20.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:20.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:20.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:20.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:20.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:20.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:20.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:20.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:20.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:20.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:20.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:20.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:20.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:20.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:20.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:20.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:20.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:20.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:20.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:20.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:20.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:20.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:20.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:20.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:20.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:20.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:20.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:20.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:20.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:20.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:20.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:20.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:20.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:20.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:20.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:20.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:20.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:20.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:20.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:20.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:20.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:20.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:20.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:20.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:20.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:20.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:20.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:20.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:20.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:20.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:20.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:20.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:20 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:21.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:21.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: pgmap v22: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:24:21.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:21.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:21.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:21.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:21.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:21.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:21.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:21.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:21.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:21.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:21.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:21.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:21.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:21.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:21.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:21.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:21.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:21.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:21.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:21.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:21.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:21.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:21.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:21.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:21.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:21.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:21.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:21.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:21.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:21.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:21.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:21.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:21.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:21.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:21.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:21.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:21.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:21.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:21.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:21.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:21.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:21.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:21.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:21.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:21.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:21.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:21.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:21.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:21.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:21.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:21.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:21.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:21.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:21.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:21.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:20 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:22.090 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:22.090 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:22.090 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:22.090 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:22.091 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:22.091 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:22.091 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:22.091 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:22.091 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:22.091 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:22.091 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:22.091 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:22.091 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:22.091 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:22.091 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:22.091 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:22.091 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:22.091 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:22.091 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:22.092 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:22.092 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:22.092 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:22.092 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:22.092 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:22.092 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:22.092 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:22.092 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:22.092 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:22.092 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:22.092 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:22.092 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:22.092 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:22.092 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:22.092 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:22.092 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:22.093 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:22.093 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:22.093 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:22.093 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:22.093 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:22.093 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:22.093 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:22.093 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:22.093 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:22.093 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:22.093 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:22.093 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:22.093 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:22.093 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:22.093 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:22.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:22.094 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:22.094 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:22.094 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:22.094 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:22.094 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:22.095 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:22.095 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:22.095 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:22.095 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:22.095 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:22.095 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:22.095 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:22.095 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:22.095 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:22.095 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:22.095 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:22.095 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:22.095 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:22.096 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:22.096 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:22.096 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:22.096 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:22.096 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:22.096 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:22.096 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:22.096 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:22.096 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:22.096 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:22.096 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:22.096 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:22.097 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:22.097 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:22.097 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:22.097 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:22.097 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:22.098 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:22.098 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:22.098 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:22.098 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:22.098 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:22.098 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:22.098 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:22.098 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:22.098 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:22.098 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:22.098 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:22.098 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:22.098 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:22.098 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:22.099 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:22.099 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:22.099 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:22.099 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:21 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:22.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:22.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:22.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:22.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:22.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:22.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:22.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:22.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:22.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:22.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:22.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:22.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:22.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:22.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:22.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:22.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:22.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:22.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:22.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:22.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:22.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:22.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:22.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:22.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:22.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:22.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:22.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:22.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:22.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:22.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:22.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:22.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:22.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:22.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:22.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:22.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:22.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:22.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:22.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:22.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:22.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:22.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:22.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:22.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:22.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:22.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:22.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:22.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:22.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:22.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:22.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:22.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:21 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:22.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:22.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[90553]: pgmap v23: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:24:22.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:22.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:22.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:22.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:22.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:22.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:22.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:22.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:22.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:22.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:22.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:22.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:22.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:22.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:22.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:22.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:22.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:22.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:22.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:22.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:22.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:22.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:22.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:22.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:22.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:22.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:22.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:22.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:22.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:22.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[93609]: pgmap v23: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:24:22.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:22.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:22.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:22.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:22.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:22.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:22.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:22.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:22.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:22.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:22.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:22.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:22.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:22.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:22.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:22.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:22.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:22.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:22.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:22.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:22.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:22.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:22.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:22.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:22.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:22.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:22.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:22.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:22.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:22.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:22.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:22.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:22.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:22.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:22.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:22.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:22.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:22.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:22.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:22.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:22.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:22.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:22.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:22.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:22.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:22.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:22.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:22.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:22.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:22.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:22.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:22.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:22.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:22.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:22.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:22.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:22.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:22.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:22.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:22.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:22.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:22.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:22.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:22.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:22.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:22.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:22.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:22.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:22.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:22.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:22 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:23.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:22 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:23.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:22 smithi139 ceph-mon[25719]: pgmap v23: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:24:23.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:22 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:23.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:22 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:23.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:22 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:23.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:22 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:23.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:22 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:23.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:22 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:23.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:22 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:23.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:22 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:23.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:22 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:23.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:22 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:23.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:22 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:23.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:22 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:23.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:22 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:23.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:22 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:23.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:22 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:23.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:22 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:23.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:22 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:23.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:22 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:23.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:22 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:23.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:22 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:23.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:22 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:23.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:22 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:23.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:22 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:23.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:22 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:23.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:22 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:23.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:22 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:23.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:22 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:23.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:22 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:23.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:22 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:23.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:22 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:23.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:22 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:23.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:22 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:23.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:22 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:23.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:22 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:23.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:22 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:23.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:22 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:23.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:22 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:23.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:22 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:23.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:22 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:23.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:22 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:23.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:22 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:23.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:22 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:23.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:22 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:23.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:22 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:23.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:22 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:23.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:22 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:23.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:22 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:23.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:22 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:23.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:22 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:24.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:24.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:24:24.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:24.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:24.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:24.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:24.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:24.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:24.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:24.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:24.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:24.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:24.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:24.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:24.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:24.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:24.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:24.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:24.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:24.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:24.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:24.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:24.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:24.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:24.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:24.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:24.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:24.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:24.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:24.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:24.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:24.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:24.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:24.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:24.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:24.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:24.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:24.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:24.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:24.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:24.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:24.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:24.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:24.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:24.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:24.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:24.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:24.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:24.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:24.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:24.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:24.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:24.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:24.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:24.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:24.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:24.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:24.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:24.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:24.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:24.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:24.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:24.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:24.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:23 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:24.140 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:24.140 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:24:24.140 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:24.140 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:24.140 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:24.140 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:24.140 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:24.140 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:24.140 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:24.141 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:24.141 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:24.141 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:24.141 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:24.141 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:24.141 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:24.141 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:24.141 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:24.141 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:24.141 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:24.141 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:24.141 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:24.141 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:24.141 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:24.142 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:24.142 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:24.142 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:24.142 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:24.142 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:24.142 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:24.142 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:24.142 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:24.142 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:24:24.143 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:24.143 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:24.143 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:24.143 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:24.143 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:24.143 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:24.143 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:24.143 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:24.143 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:24.143 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:24.143 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:24.143 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:24.143 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:24.143 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:24.144 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:24.144 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:24.144 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:24.144 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:24.144 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:24.144 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:24.144 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:24.144 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:24.144 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:24.144 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:24.144 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:24.144 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:24.144 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:24.144 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:24.145 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:24.145 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:24.145 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:24.145 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:24.146 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:24.146 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:24.146 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:24.146 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:24.146 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:24.147 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:24.147 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:24.147 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:24.147 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:24.147 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:24.147 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:24.147 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:24.147 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:24.147 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:24.147 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:24.147 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:24.147 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:24.147 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:24.147 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:24.147 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:24.148 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:24.148 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:24.148 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:24.148 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:24.148 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:24.148 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:24.148 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:24.148 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:24.148 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:24.148 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:24.149 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:24.149 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:24.149 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:24.149 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:24.149 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:24.149 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:24.149 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:24.149 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:24.149 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:24.149 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:24.149 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:24.149 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:24.149 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:24.149 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:24.149 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:24.150 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:24.150 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:24.150 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:24.150 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:24.150 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:24.150 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:24.150 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:24.150 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:24.150 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:24.150 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:24.151 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:24.151 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:24.151 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:24.151 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:24.151 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:24.151 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:24.151 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:24.151 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:24.151 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:25.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: pgmap v24: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:24:25.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:25.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:25.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:25.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:25.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:25.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:25.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:25.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:25.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:25.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:25.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:25.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:25.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:25.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:25.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:25.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:25.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:25.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:25.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:25.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:25.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:25.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:25.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:25.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:25.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:25.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:25.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:25.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:25.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:25.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:25.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:25.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:25.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:25.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:25.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:25.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:25.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:25.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:25.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:25.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:25.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:25.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:25.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:25.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:25.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:24 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.136 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: pgmap v24: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:24:25.136 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.136 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.137 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.137 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.137 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.137 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.137 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.137 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.137 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.137 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.137 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:25.137 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:25.137 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:25.137 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:25.137 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.137 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.138 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:25.138 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:25.138 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:25.138 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:25.138 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.138 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:25.138 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:25.138 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:25.138 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:25.138 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.138 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:25.138 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:25.138 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:25.139 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: pgmap v24: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:24:25.139 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.139 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.139 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.139 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.139 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.139 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.139 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.139 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.139 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.139 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.140 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:25.140 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:25.140 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:25.140 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:25.140 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.140 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.140 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:25.140 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:25.140 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:25.140 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:25.140 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.140 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:25.140 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:25.140 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:25.140 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:25.141 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.141 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:25.141 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:25.141 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:25.141 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:25.141 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.141 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:25.141 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:25.141 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:25.142 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:25.142 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.142 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:25.142 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:25.142 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:25.142 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:25.142 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.142 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:25.142 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:25.142 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:25.142 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:25.142 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.142 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:25.142 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:25.143 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:25.143 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:25.143 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.143 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:25.143 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:25.143 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:25.143 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:25.143 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.144 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:25.144 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:25.144 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:25.144 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:25.144 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.144 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:25.144 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:25.144 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:25.144 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:25.144 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.145 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:25.145 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.145 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:25.145 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:25.145 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:25.145 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:25.145 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.145 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:25.145 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:25.145 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:25.145 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:25.145 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.145 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:25.146 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:25.146 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:25.146 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:25.146 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.146 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:25.146 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:25.146 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:25.146 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:25.146 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.146 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:25.146 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:25.146 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:25.146 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:25.147 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.147 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:25.147 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:25.147 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:25.147 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:25.147 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.147 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:25.147 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:25.147 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:25.147 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:25.147 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:25.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:25.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:25.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:25.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:25.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:25.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:25.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:25.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:25.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:25.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:25.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:25.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:25.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:25.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:25.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:25.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:25.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:25.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:25.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:25.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:25.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:25.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:25.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:25.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:25.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:25.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:25.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:25.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:25.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:25.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:25.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:25.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:25.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:25.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:25.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:25.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:25.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:25.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:25.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:25.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:25.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:25.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:25.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:25.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:25.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:25.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:25.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-24T11:24:25.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mon ok-to-stop", "ids": ["c"]}]: dispatch 2024-03-24T11:24:25.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:25.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:25.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:25.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:25.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:25.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:25.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:25.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:25.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:25.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:25.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:25.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:25.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:25.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:25.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:25.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-24T11:24:25.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:25 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mon ok-to-stop", "ids": ["c"]}]: dispatch 2024-03-24T11:24:26.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:26.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:26.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:26.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:26.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:26.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:26.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:26.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:26.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:26.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:26.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:26.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:26.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:26.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:26.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:26.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:26.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:26.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:26.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:26.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:26.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:26.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:26.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:26.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:26.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:26.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:26.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:26.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:26.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:26.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:26.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:26.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:26.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:26.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:26.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:26.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:26.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:26.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:26.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:26.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:26.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:26.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:26.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:26.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:26.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:26.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:26.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:26.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:26.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:26.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:26.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:26.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:26.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-24T11:24:26.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:25 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mon ok-to-stop", "ids": ["c"]}]: dispatch 2024-03-24T11:24:27.178 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:26 smithi062 ceph-mon[90553]: pgmap v25: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:24:27.178 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:26 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:27.178 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:26 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:27.178 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:26 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:27.178 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:26 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:27.178 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:26 smithi062 ceph-mon[90553]: Upgrade: It appears safe to stop mon.c 2024-03-24T11:24:27.178 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:24:27.179 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:26 smithi062 ceph-mon[93609]: pgmap v25: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:24:27.179 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:26 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:27.179 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:26 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:27.179 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:26 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:27.179 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:26 smithi062 ceph-mon[93609]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:27.179 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:26 smithi062 ceph-mon[93609]: Upgrade: It appears safe to stop mon.c 2024-03-24T11:24:27.179 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:26 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:24:27.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:26 smithi139 ceph-mon[25719]: pgmap v25: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:24:27.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:26 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:27.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:26 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:27.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:26 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:27.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:26 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:27.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:26 smithi139 ceph-mon[25719]: Upgrade: It appears safe to stop mon.c 2024-03-24T11:24:27.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:26 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:24:28.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T11:24:28.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T11:24:28.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:28.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:27 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T11:24:28.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:27 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T11:24:28.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:27 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:28.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:27 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T11:24:28.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:27 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T11:24:28.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:27 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:28.965 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:24:28 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:24:28.659Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"smithi062:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"172.21.15.139:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-24T11:24:29.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:28 smithi062 ceph-mon[90553]: pgmap v26: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:24:29.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:28 smithi062 ceph-mon[90553]: Upgrade: Updating mon.c 2024-03-24T11:24:29.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:28 smithi062 ceph-mon[90553]: Deploying daemon mon.c on smithi062 2024-03-24T11:24:29.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:29.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:29.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:29.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:29.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:29.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:29.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:28 smithi062 ceph-mon[93609]: pgmap v26: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:24:29.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:28 smithi062 ceph-mon[93609]: Upgrade: Updating mon.c 2024-03-24T11:24:29.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:28 smithi062 ceph-mon[93609]: Deploying daemon mon.c on smithi062 2024-03-24T11:24:29.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:28 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:29.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:28 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:29.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:28 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:29.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:28 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:29.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:28 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:29.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:28 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:29.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:28 smithi139 ceph-mon[25719]: pgmap v26: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:24:29.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:28 smithi139 ceph-mon[25719]: Upgrade: Updating mon.c 2024-03-24T11:24:29.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:28 smithi139 ceph-mon[25719]: Deploying daemon mon.c on smithi062 2024-03-24T11:24:29.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:28 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:29.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:28 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:29.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:28 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:29.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:28 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:29.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:28 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:29.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:28 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:30.966 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:24:30 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:24:30] "GET /metrics HTTP/1.1" 200 38330 "" "Prometheus/2.33.4" 2024-03-24T11:24:30.966 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:30 smithi062 systemd[1]: Stopping Ceph mon.c for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:24:31.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:30 smithi062 ceph-mon[90553]: pgmap v27: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:24:31.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:31.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:31.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:31.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:30 smithi062 ceph-mon[93609]: pgmap v27: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:24:31.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:30 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:31.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:30 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:31.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:30 smithi062 ceph-mon[93609]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:31.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:31 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-c[93605]: 2024-03-24T11:24:31.078+0000 7f4f8746b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T11:24:31.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:31 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-c[93605]: 2024-03-24T11:24:31.078+0000 7f4f8746b700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-03-24T11:24:31.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:31 smithi062 podman[98481]: 2024-03-24 11:24:31.119586543 +0000 UTC m=+0.148805018 container died fe870b225563ceb06347bd74a26a9136b9dd8f6b398ae0beda5ca69a0149bdce (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-c, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-db0330b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307) 2024-03-24T11:24:31.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:30 smithi139 ceph-mon[25719]: pgmap v27: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:24:31.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:30 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:31.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:30 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:31.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:30 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:31.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:31 smithi062 podman[98481]: 2024-03-24 11:24:31.486915873 +0000 UTC m=+0.516134355 container cleanup fe870b225563ceb06347bd74a26a9136b9dd8f6b398ae0beda5ca69a0149bdce (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T11:24:31.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:31 smithi062 bash[98481]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-c 2024-03-24T11:24:32.011 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:31 smithi062 podman[98495]: 2024-03-24 11:24:31.738475384 +0000 UTC m=+0.614637434 container remove fe870b225563ceb06347bd74a26a9136b9dd8f6b398ae0beda5ca69a0149bdce (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-c, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-db0330b, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True) 2024-03-24T11:24:32.011 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:31 smithi062 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mon.c.service: Deactivated successfully. 2024-03-24T11:24:32.011 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:31 smithi062 systemd[1]: Stopped Ceph mon.c for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:24:32.011 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:31 smithi062 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mon.c.service: Consumed 1.252s CPU time. 2024-03-24T11:24:32.314 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 systemd[1]: Starting Ceph mon.c for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:24:32.582 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 podman[98624]: 2024-03-24 11:24:32.314892143 +0000 UTC m=+0.105095765 container create 459e7449ec3927a943e5a6d4eb9682d730abe01433b21ef9dc53de0ce9027d0d (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240307, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-db0330b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T11:24:32.582 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 podman[98624]: 2024-03-24 11:24:32.227847269 +0000 UTC m=+0.018050895 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:24:32.582 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 podman[98624]: 2024-03-24 11:24:32.556097111 +0000 UTC m=+0.346300744 container init 459e7449ec3927a943e5a6d4eb9682d730abe01433b21ef9dc53de0ce9027d0d (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-c, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-db0330b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T11:24:32.582 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 podman[98624]: 2024-03-24 11:24:32.559388062 +0000 UTC m=+0.349591687 container start 459e7449ec3927a943e5a6d4eb9682d730abe01433b21ef9dc53de0ce9027d0d (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-db0330b) 2024-03-24T11:24:32.863 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: set uid:gid to 167:167 (ceph:ceph) 2024-03-24T11:24:32.864 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev), process ceph-mon, pid 2 2024-03-24T11:24:32.864 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: pidfile_write: ignore empty --pid-file 2024-03-24T11:24:32.864 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: load: jerasure load: lrc 2024-03-24T11:24:32.864 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: RocksDB version: 7.9.2 2024-03-24T11:24:32.864 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Git sha 0 2024-03-24T11:24:32.864 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Compile date 2024-03-16 10:36:11 2024-03-24T11:24:32.864 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: DB SUMMARY 2024-03-24T11:24:32.864 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: DB Session ID: 6EOIJTG7YC9CQD90YB0F 2024-03-24T11:24:32.864 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: CURRENT file: CURRENT 2024-03-24T11:24:32.864 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: IDENTITY file: IDENTITY 2024-03-24T11:24:32.864 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: MANIFEST file: MANIFEST-000047 size: 277 Bytes 2024-03-24T11:24:32.864 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 2, files: 000042.sst 000045.sst 2024-03-24T11:24:32.865 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000046.log size: 1860623 ; 2024-03-24T11:24:32.865 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.error_if_exists: 0 2024-03-24T11:24:32.865 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.create_if_missing: 0 2024-03-24T11:24:32.865 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.paranoid_checks: 1 2024-03-24T11:24:32.865 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-24T11:24:32.865 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-24T11:24:32.865 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-24T11:24:32.865 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.env: 0x55924b207ae0 2024-03-24T11:24:32.865 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.fs: PosixFileSystem 2024-03-24T11:24:32.865 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.info_log: 0x55924ca1b8c0 2024-03-24T11:24:32.865 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.max_file_opening_threads: 16 2024-03-24T11:24:32.865 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.statistics: (nil) 2024-03-24T11:24:32.865 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.use_fsync: 0 2024-03-24T11:24:32.865 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.max_log_file_size: 0 2024-03-24T11:24:32.866 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-24T11:24:32.866 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-24T11:24:32.866 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.keep_log_file_num: 1000 2024-03-24T11:24:32.866 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.recycle_log_file_num: 0 2024-03-24T11:24:32.866 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.allow_fallocate: 1 2024-03-24T11:24:32.866 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.allow_mmap_reads: 0 2024-03-24T11:24:32.866 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.allow_mmap_writes: 0 2024-03-24T11:24:32.866 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.use_direct_reads: 0 2024-03-24T11:24:32.866 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-24T11:24:32.866 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.create_missing_column_families: 0 2024-03-24T11:24:32.866 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.db_log_dir: 2024-03-24T11:24:32.866 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.wal_dir: 2024-03-24T11:24:32.866 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-24T11:24:32.867 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-24T11:24:32.867 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-24T11:24:32.867 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-24T11:24:32.867 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-24T11:24:32.867 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-24T11:24:32.867 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.advise_random_on_open: 1 2024-03-24T11:24:32.867 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.db_write_buffer_size: 0 2024-03-24T11:24:32.867 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.write_buffer_manager: 0x55924c9eedc0 2024-03-24T11:24:32.867 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-24T11:24:32.867 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-24T11:24:32.867 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-24T11:24:32.867 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.rate_limiter: (nil) 2024-03-24T11:24:32.867 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-24T11:24:32.867 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.wal_recovery_mode: 2 2024-03-24T11:24:32.867 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.enable_thread_tracking: 0 2024-03-24T11:24:32.868 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.enable_pipelined_write: 0 2024-03-24T11:24:32.868 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.unordered_write: 0 2024-03-24T11:24:32.868 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-24T11:24:32.868 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-24T11:24:32.868 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-24T11:24:32.868 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-24T11:24:32.868 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.row_cache: None 2024-03-24T11:24:32.868 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.wal_filter: None 2024-03-24T11:24:32.868 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-24T11:24:32.868 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.allow_ingest_behind: 0 2024-03-24T11:24:32.868 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.two_write_queues: 0 2024-03-24T11:24:32.868 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.manual_wal_flush: 0 2024-03-24T11:24:32.868 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.wal_compression: 0 2024-03-24T11:24:32.868 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.atomic_flush: 0 2024-03-24T11:24:32.868 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-24T11:24:32.869 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-24T11:24:32.869 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-24T11:24:32.869 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.log_readahead_size: 0 2024-03-24T11:24:32.869 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-24T11:24:32.869 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.best_efforts_recovery: 0 2024-03-24T11:24:32.869 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-24T11:24:32.869 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-24T11:24:32.869 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.allow_data_in_errors: 0 2024-03-24T11:24:32.869 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.db_host_id: __hostname__ 2024-03-24T11:24:32.870 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-24T11:24:32.870 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.max_background_jobs: 2 2024-03-24T11:24:32.871 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.max_background_compactions: -1 2024-03-24T11:24:32.871 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.max_subcompactions: 1 2024-03-24T11:24:32.871 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-24T11:24:32.871 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-24T11:24:32.871 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-24T11:24:32.871 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.max_total_wal_size: 0 2024-03-24T11:24:32.871 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-24T11:24:32.871 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-24T11:24:32.871 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-24T11:24:32.871 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-24T11:24:32.871 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.max_open_files: -1 2024-03-24T11:24:32.871 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.bytes_per_sync: 0 2024-03-24T11:24:32.871 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-24T11:24:32.871 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-24T11:24:32.871 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.compaction_readahead_size: 0 2024-03-24T11:24:32.872 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.max_background_flushes: -1 2024-03-24T11:24:32.872 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Compression algorithms supported: 2024-03-24T11:24:32.872 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-24T11:24:32.872 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: kZSTD supported: 0 2024-03-24T11:24:32.872 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: kXpressCompression supported: 0 2024-03-24T11:24:32.872 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: kLZ4HCCompression supported: 1 2024-03-24T11:24:32.872 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: kZlibCompression supported: 1 2024-03-24T11:24:32.872 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: kSnappyCompression supported: 1 2024-03-24T11:24:32.872 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: kLZ4Compression supported: 1 2024-03-24T11:24:32.872 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: kBZip2Compression supported: 0 2024-03-24T11:24:32.872 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-24T11:24:32.872 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-24T11:24:32.872 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000047 2024-03-24T11:24:32.872 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-24T11:24:32.872 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-24T11:24:32.873 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.merge_operator: 2024-03-24T11:24:32.873 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.compaction_filter: None 2024-03-24T11:24:32.873 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.compaction_filter_factory: None 2024-03-24T11:24:32.873 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.sst_partitioner_factory: None 2024-03-24T11:24:32.873 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-24T11:24:32.873 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-24T11:24:32.873 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55924ca1b9c0) 2024-03-24T11:24:32.873 INFO:journalctl@ceph.mon.c.smithi062.stdout: cache_index_and_filter_blocks: 1 2024-03-24T11:24:32.873 INFO:journalctl@ceph.mon.c.smithi062.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-24T11:24:32.873 INFO:journalctl@ceph.mon.c.smithi062.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-24T11:24:32.873 INFO:journalctl@ceph.mon.c.smithi062.stdout: pin_top_level_index_and_filter: 1 2024-03-24T11:24:32.874 INFO:journalctl@ceph.mon.c.smithi062.stdout: index_type: 0 2024-03-24T11:24:32.874 INFO:journalctl@ceph.mon.c.smithi062.stdout: data_block_index_type: 0 2024-03-24T11:24:32.874 INFO:journalctl@ceph.mon.c.smithi062.stdout: index_shortening: 1 2024-03-24T11:24:32.874 INFO:journalctl@ceph.mon.c.smithi062.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-24T11:24:32.874 INFO:journalctl@ceph.mon.c.smithi062.stdout: checksum: 4 2024-03-24T11:24:32.874 INFO:journalctl@ceph.mon.c.smithi062.stdout: no_block_cache: 0 2024-03-24T11:24:32.874 INFO:journalctl@ceph.mon.c.smithi062.stdout: block_cache: 0x55924ca1f090 2024-03-24T11:24:32.874 INFO:journalctl@ceph.mon.c.smithi062.stdout: block_cache_name: BinnedLRUCache 2024-03-24T11:24:32.874 INFO:journalctl@ceph.mon.c.smithi062.stdout: block_cache_options: 2024-03-24T11:24:32.874 INFO:journalctl@ceph.mon.c.smithi062.stdout: capacity : 536870912 2024-03-24T11:24:32.874 INFO:journalctl@ceph.mon.c.smithi062.stdout: num_shard_bits : 4 2024-03-24T11:24:32.874 INFO:journalctl@ceph.mon.c.smithi062.stdout: strict_capacity_limit : 0 2024-03-24T11:24:32.874 INFO:journalctl@ceph.mon.c.smithi062.stdout: high_pri_pool_ratio: 0.000 2024-03-24T11:24:32.874 INFO:journalctl@ceph.mon.c.smithi062.stdout: block_cache_compressed: (nil) 2024-03-24T11:24:32.875 INFO:journalctl@ceph.mon.c.smithi062.stdout: persistent_cache: (nil) 2024-03-24T11:24:32.875 INFO:journalctl@ceph.mon.c.smithi062.stdout: block_size: 4096 2024-03-24T11:24:32.875 INFO:journalctl@ceph.mon.c.smithi062.stdout: block_size_deviation: 10 2024-03-24T11:24:32.875 INFO:journalctl@ceph.mon.c.smithi062.stdout: block_restart_interval: 16 2024-03-24T11:24:32.875 INFO:journalctl@ceph.mon.c.smithi062.stdout: index_block_restart_interval: 1 2024-03-24T11:24:32.875 INFO:journalctl@ceph.mon.c.smithi062.stdout: metadata_block_size: 4096 2024-03-24T11:24:32.875 INFO:journalctl@ceph.mon.c.smithi062.stdout: partition_filters: 0 2024-03-24T11:24:32.875 INFO:journalctl@ceph.mon.c.smithi062.stdout: use_delta_encoding: 1 2024-03-24T11:24:32.875 INFO:journalctl@ceph.mon.c.smithi062.stdout: filter_policy: bloomfilter 2024-03-24T11:24:32.875 INFO:journalctl@ceph.mon.c.smithi062.stdout: whole_key_filtering: 1 2024-03-24T11:24:32.875 INFO:journalctl@ceph.mon.c.smithi062.stdout: verify_compression: 0 2024-03-24T11:24:32.875 INFO:journalctl@ceph.mon.c.smithi062.stdout: read_amp_bytes_per_bit: 0 2024-03-24T11:24:32.875 INFO:journalctl@ceph.mon.c.smithi062.stdout: format_version: 5 2024-03-24T11:24:32.875 INFO:journalctl@ceph.mon.c.smithi062.stdout: enable_index_compression: 1 2024-03-24T11:24:32.875 INFO:journalctl@ceph.mon.c.smithi062.stdout: block_align: 0 2024-03-24T11:24:32.875 INFO:journalctl@ceph.mon.c.smithi062.stdout: max_auto_readahead_size: 262144 2024-03-24T11:24:32.876 INFO:journalctl@ceph.mon.c.smithi062.stdout: prepopulate_block_cache: 0 2024-03-24T11:24:32.876 INFO:journalctl@ceph.mon.c.smithi062.stdout: initial_auto_readahead_size: 8192 2024-03-24T11:24:32.876 INFO:journalctl@ceph.mon.c.smithi062.stdout: num_file_reads_for_auto_readahead: 2 2024-03-24T11:24:32.876 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.write_buffer_size: 33554432 2024-03-24T11:24:32.876 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.max_write_buffer_number: 2 2024-03-24T11:24:32.876 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.compression: NoCompression 2024-03-24T11:24:32.876 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.bottommost_compression: Disabled 2024-03-24T11:24:32.876 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.prefix_extractor: nullptr 2024-03-24T11:24:32.876 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-24T11:24:32.876 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.num_levels: 7 2024-03-24T11:24:32.876 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-24T11:24:32.876 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-24T11:24:32.876 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-24T11:24:32.876 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-24T11:24:32.876 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-24T11:24:32.877 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-24T11:24:32.877 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-24T11:24:32.877 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-24T11:24:32.877 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-24T11:24:32.877 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-24T11:24:32.877 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-24T11:24:32.877 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-24T11:24:32.877 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-24T11:24:32.877 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.compression_opts.level: 32767 2024-03-24T11:24:32.877 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.compression_opts.strategy: 0 2024-03-24T11:24:32.877 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-24T11:24:32.878 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-24T11:24:32.878 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-24T11:24:32.878 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-24T11:24:32.878 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.compression_opts.enabled: false 2024-03-24T11:24:32.878 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-24T11:24:32.878 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-24T11:24:32.878 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-24T11:24:32.878 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-24T11:24:32.878 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.target_file_size_base: 67108864 2024-03-24T11:24:32.878 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-24T11:24:32.878 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-24T11:24:32.878 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-24T11:24:32.878 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-24T11:24:32.878 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-24T11:24:32.879 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-24T11:24:32.879 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-24T11:24:32.879 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-24T11:24:32.879 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-24T11:24:32.879 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-24T11:24:32.879 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-24T11:24:32.879 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-24T11:24:32.879 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-24T11:24:32.879 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-24T11:24:32.879 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.arena_block_size: 1048576 2024-03-24T11:24:32.879 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-24T11:24:32.879 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-24T11:24:32.879 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.disable_auto_compactions: 0 2024-03-24T11:24:32.879 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-24T11:24:32.879 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-24T11:24:32.880 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-24T11:24:32.880 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-24T11:24:32.880 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-24T11:24:32.880 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-24T11:24:32.880 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-24T11:24:32.880 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-24T11:24:32.880 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-24T11:24:32.880 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-24T11:24:32.880 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.table_properties_collectors: 2024-03-24T11:24:32.880 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.inplace_update_support: 0 2024-03-24T11:24:32.880 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-24T11:24:32.880 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-24T11:24:32.880 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-24T11:24:32.880 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-24T11:24:32.880 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.bloom_locality: 0 2024-03-24T11:24:32.881 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.max_successive_merges: 0 2024-03-24T11:24:32.881 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-24T11:24:32.881 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.paranoid_file_checks: 0 2024-03-24T11:24:32.881 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.force_consistency_checks: 1 2024-03-24T11:24:32.881 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.report_bg_io_stats: 0 2024-03-24T11:24:32.881 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.ttl: 2592000 2024-03-24T11:24:32.881 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-24T11:24:32.881 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-24T11:24:32.881 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-24T11:24:32.881 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.enable_blob_files: false 2024-03-24T11:24:32.881 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.min_blob_size: 0 2024-03-24T11:24:32.881 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.blob_file_size: 268435456 2024-03-24T11:24:32.881 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-24T11:24:32.881 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-24T11:24:32.881 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-24T11:24:32.882 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-24T11:24:32.882 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-24T11:24:32.882 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.blob_file_starting_level: 0 2024-03-24T11:24:32.882 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-24T11:24:32.882 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 42.sst 2024-03-24T11:24:32.882 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-24T11:24:32.882 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000047 succeeded,manifest_file_number is 47, next_file_number is 49, last_sequence is 26570, log_number is 41,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 41 2024-03-24T11:24:32.882 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 41 2024-03-24T11:24:32.882 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: b6db2122-ff52-4d85-81c5-579d4a8837d2 2024-03-24T11:24:32.882 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711279472588757, "job": 1, "event": "recovery_started", "wal_files": [46]} 2024-03-24T11:24:32.883 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #46 mode 2 2024-03-24T11:24:32.883 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711279472594683, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 50, "file_size": 1145288, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 26575, "largest_seqno": 28625, "table_properties": {"data_size": 1138590, "index_size": 2674, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 3013, "raw_key_size": 26807, "raw_average_key_size": 22, "raw_value_size": 1119856, "raw_average_value_size": 939, "num_data_blocks": 108, "num_entries": 1192, "num_filter_entries": 1192, "num_deletions": 259, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711279472, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "b6db2122-ff52-4d85-81c5-579d4a8837d2", "db_session_id": "6EOIJTG7YC9CQD90YB0F", "orig_file_number": 50, "seqno_to_time_mapping": "N/A"}} 2024-03-24T11:24:32.883 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711279472594790, "job": 1, "event": "recovery_finished"} 2024-03-24T11:24:32.883 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: [db/version_set.cc:5047] Creating manifest 52 2024-03-24T11:24:32.883 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-24T11:24:32.883 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000046.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-24T11:24:32.883 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55924cb10000 2024-03-24T11:24:32.883 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: DB pointer 0x55924cafa000 2024-03-24T11:24:32.883 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-24T11:24:32.883 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-24T11:24:32.883 INFO:journalctl@ceph.mon.c.smithi062.stdout: ** DB Stats ** 2024-03-24T11:24:32.883 INFO:journalctl@ceph.mon.c.smithi062.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-24T11:24:32.883 INFO:journalctl@ceph.mon.c.smithi062.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-24T11:24:32.883 INFO:journalctl@ceph.mon.c.smithi062.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-24T11:24:32.883 INFO:journalctl@ceph.mon.c.smithi062.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-24T11:24:32.884 INFO:journalctl@ceph.mon.c.smithi062.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-24T11:24:32.884 INFO:journalctl@ceph.mon.c.smithi062.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-24T11:24:32.884 INFO:journalctl@ceph.mon.c.smithi062.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-24T11:24:32.884 INFO:journalctl@ceph.mon.c.smithi062.stdout: 2024-03-24T11:24:32.884 INFO:journalctl@ceph.mon.c.smithi062.stdout: ** Compaction Stats [default] ** 2024-03-24T11:24:32.884 INFO:journalctl@ceph.mon.c.smithi062.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-24T11:24:32.884 INFO:journalctl@ceph.mon.c.smithi062.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-24T11:24:32.884 INFO:journalctl@ceph.mon.c.smithi062.stdout: L0 2/0 1.65 MB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 401.0 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-03-24T11:24:32.884 INFO:journalctl@ceph.mon.c.smithi062.stdout: L6 1/0 11.02 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-03-24T11:24:32.884 INFO:journalctl@ceph.mon.c.smithi062.stdout: Sum 3/0 12.68 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 401.0 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-03-24T11:24:32.884 INFO:journalctl@ceph.mon.c.smithi062.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 401.0 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-03-24T11:24:32.884 INFO:journalctl@ceph.mon.c.smithi062.stdout: 2024-03-24T11:24:32.884 INFO:journalctl@ceph.mon.c.smithi062.stdout: ** Compaction Stats [default] ** 2024-03-24T11:24:32.884 INFO:journalctl@ceph.mon.c.smithi062.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-24T11:24:32.884 INFO:journalctl@ceph.mon.c.smithi062.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-24T11:24:32.885 INFO:journalctl@ceph.mon.c.smithi062.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 401.0 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-03-24T11:24:32.885 INFO:journalctl@ceph.mon.c.smithi062.stdout: 2024-03-24T11:24:32.885 INFO:journalctl@ceph.mon.c.smithi062.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-24T11:24:32.885 INFO:journalctl@ceph.mon.c.smithi062.stdout: 2024-03-24T11:24:32.885 INFO:journalctl@ceph.mon.c.smithi062.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-24T11:24:32.885 INFO:journalctl@ceph.mon.c.smithi062.stdout: Flush(GB): cumulative 0.001, interval 0.001 2024-03-24T11:24:32.885 INFO:journalctl@ceph.mon.c.smithi062.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-24T11:24:32.885 INFO:journalctl@ceph.mon.c.smithi062.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-24T11:24:32.885 INFO:journalctl@ceph.mon.c.smithi062.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-24T11:24:32.885 INFO:journalctl@ceph.mon.c.smithi062.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-24T11:24:32.885 INFO:journalctl@ceph.mon.c.smithi062.stdout: Cumulative compaction: 0.00 GB write, 105.27 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-24T11:24:32.885 INFO:journalctl@ceph.mon.c.smithi062.stdout: Interval compaction: 0.00 GB write, 105.27 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-24T11:24:32.885 INFO:journalctl@ceph.mon.c.smithi062.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-24T11:24:32.885 INFO:journalctl@ceph.mon.c.smithi062.stdout: Block cache BinnedLRUCache@0x55924ca1f090#2 capacity: 512.00 MB usage: 8.47 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-03-24T11:24:32.886 INFO:journalctl@ceph.mon.c.smithi062.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,4.19 KB,0.000798702%) IndexBlock(2,4.28 KB,0.000816584%) Misc(1,0.00 KB,0%) 2024-03-24T11:24:32.886 INFO:journalctl@ceph.mon.c.smithi062.stdout: 2024-03-24T11:24:32.886 INFO:journalctl@ceph.mon.c.smithi062.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-24T11:24:32.886 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: starting mon.c rank 1 at public addrs [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] at bind addrs [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] mon_data /var/lib/ceph/mon/ceph-c fsid ebb89568-e9ce-11ee-95cd-87774f69a715 2024-03-24T11:24:32.886 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: mon.c@-1(???) e3 preinit fsid ebb89568-e9ce-11ee-95cd-87774f69a715 2024-03-24T11:24:32.886 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: mon.c@-1(???).mds e1 new map 2024-03-24T11:24:32.886 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: mon.c@-1(???).mds e1 print_map 2024-03-24T11:24:32.886 INFO:journalctl@ceph.mon.c.smithi062.stdout: e1 2024-03-24T11:24:32.886 INFO:journalctl@ceph.mon.c.smithi062.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T11:24:32.886 INFO:journalctl@ceph.mon.c.smithi062.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T11:24:32.886 INFO:journalctl@ceph.mon.c.smithi062.stdout: legacy client fscid: -1 2024-03-24T11:24:32.886 INFO:journalctl@ceph.mon.c.smithi062.stdout: 2024-03-24T11:24:32.886 INFO:journalctl@ceph.mon.c.smithi062.stdout: No filesystems configured 2024-03-24T11:24:32.886 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: mon.c@-1(???).osd e108 crush map has features 3314933000854323200, adjusting msgr requires 2024-03-24T11:24:32.886 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: mon.c@-1(???).osd e108 crush map has features 432629239337189376, adjusting msgr requires 2024-03-24T11:24:32.887 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: mon.c@-1(???).osd e108 crush map has features 432629239337189376, adjusting msgr requires 2024-03-24T11:24:32.887 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: mon.c@-1(???).osd e108 crush map has features 432629239337189376, adjusting msgr requires 2024-03-24T11:24:32.887 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 ceph-mon[98643]: mon.c@-1(???).paxosservice(auth 1..27) refresh upgraded, format 0 -> 3 2024-03-24T11:24:32.887 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 bash[98624]: 459e7449ec3927a943e5a6d4eb9682d730abe01433b21ef9dc53de0ce9027d0d 2024-03-24T11:24:32.887 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:32 smithi062 systemd[1]: Started Ceph mon.c for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:24:34.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:24:34.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: mon.c calling monitor election 2024-03-24T11:24:34.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: pgmap v29: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:24:34.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: mon.a calling monitor election 2024-03-24T11:24:34.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-24T11:24:34.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: monmap e3: 3 mons at {a=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0],b=[v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0],c=[v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-24T11:24:34.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: fsmap 2024-03-24T11:24:34.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: osdmap e108: 8 total, 8 up, 8 in 2024-03-24T11:24:34.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: mgrmap e44: y(active, since 52s), standbys: x 2024-03-24T11:24:34.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: overall HEALTH_OK 2024-03-24T11:24:34.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:34.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:34.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:34.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:34.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:34.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:34.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:34.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:34.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:34.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:34.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:34.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:34.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:34.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:34.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:34.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:34.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:34.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:34.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:34.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:34.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:34.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:34.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:34.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:34.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:34.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:34.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:34.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:34.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:34.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:34.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:34.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:34.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:34.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:34.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:34.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:34.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:34.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:34.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:34.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:34.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:34.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:34.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:34.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:34.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:34.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:34.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:34.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:34.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:34.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:34.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:34.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:34 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:34.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:24:34.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: mon.c calling monitor election 2024-03-24T11:24:34.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: pgmap v29: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:24:34.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: mon.a calling monitor election 2024-03-24T11:24:34.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-24T11:24:34.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: monmap e3: 3 mons at {a=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0],b=[v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0],c=[v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-24T11:24:34.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: fsmap 2024-03-24T11:24:34.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: osdmap e108: 8 total, 8 up, 8 in 2024-03-24T11:24:34.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: mgrmap e44: y(active, since 52s), standbys: x 2024-03-24T11:24:34.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: overall HEALTH_OK 2024-03-24T11:24:34.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:34.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:34.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:34.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:34.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:34.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:34.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:34.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:34.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:34.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:34.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:34.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:34.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:34.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:34.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:34.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:34.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:34.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:34.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:34.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:34.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:34.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:24:34.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: mon.c calling monitor election 2024-03-24T11:24:34.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: pgmap v29: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:24:34.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: mon.a calling monitor election 2024-03-24T11:24:34.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-24T11:24:34.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: monmap e3: 3 mons at {a=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0],b=[v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0],c=[v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-24T11:24:34.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: fsmap 2024-03-24T11:24:34.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: osdmap e108: 8 total, 8 up, 8 in 2024-03-24T11:24:34.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: mgrmap e44: y(active, since 52s), standbys: x 2024-03-24T11:24:34.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: overall HEALTH_OK 2024-03-24T11:24:34.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:34.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:34.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:34.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:34.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:34.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:34.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:34.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:34.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:34.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:34.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:34.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:34.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:34.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:34.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:34.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:34.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:34.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:34.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:34.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:34.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:34.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:34.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:34.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:34.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:34.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:34.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:34.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:34.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:34.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:34.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:34.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:34.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:34.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:34.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:34.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:34.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:34.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:34.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:34.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:34.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:34.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:34.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:34.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:34.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:34.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:34.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:34.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:34.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:34.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:34.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:34.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:34.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:34.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:34.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:34.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:34.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:34.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:34.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:34.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:34.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:34.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:34.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:34.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:34.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:34.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:34.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:34.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:34.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:34.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:34.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:34.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:34.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:34.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:34.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:34.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:34.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:34.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:34.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:34.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:34.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:34.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:34.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:35.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:35.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:35.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:35.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:35.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:35.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:35.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:35.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:35.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:35.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:35.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:35.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:35.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:35.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:35.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:35.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:35.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:35.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:35.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:35.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:35.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:35.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:35.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:35.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:35.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:35.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:35.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:35.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:35.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:35.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:35.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:35.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:35.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:35.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:35.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:35.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:35.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:35.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:35.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:35.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:35.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:35.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:35.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:35.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:35.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:35.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:35.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:35.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:35.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:35.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:35.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:35.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:35.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:35.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:35.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:35.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:35.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:35.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:35.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:35.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:35.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:35.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:35.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:35.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:35.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:35.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:35.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:35.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:35.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:35.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:35.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:35.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:35.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:35.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:35.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:35.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:35.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:35.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:35.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:35.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:35.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:35.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:35.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:35.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:35.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:35.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:35.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:35.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:35.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:35.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:35.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:35.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:35.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:35.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:35.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:35.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:35.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:35.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:35.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:35.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:35.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:35.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:35.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:35.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:35.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:35.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:35.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:35.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:35.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:35.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:35.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:35.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:35.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:35.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:35.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:35.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:35.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:35.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:35.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:35.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:35.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:35.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:35.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:35.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:35.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:35.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:35.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:35.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:35.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:35.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:35.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:35.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:35.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:35.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:35.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:35.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:35.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:35.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:35.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:35.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:35.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:35.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:35.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:35.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:35.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:35.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:35.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:35.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:35.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:35.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:35.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:35.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:35.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:35.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:35.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:35.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:35.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:35.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:35.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:35.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:35.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:35.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:35.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:35.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:35.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:35.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:35.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:35.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:35.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:35.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:35.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:35.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:35.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:35.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:35.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:35.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:35.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:35.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:35.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:35.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:35.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:35.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:35.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:35.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:35.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:35.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:35.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:35.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:35.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:35.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:35.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:35.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:35.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:35.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:35.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:35.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:35.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:35.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:35 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:36.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:36.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: pgmap v30: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:24:36.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:36.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:36.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:36.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:36.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:36.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:36.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:36.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:36.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:36.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:36.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:36.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:36.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:36.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:36.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:36.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:36.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:36.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:36.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:36.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:36.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:36.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:36.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:36.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:36.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:36.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:36.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:36.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:36.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: pgmap v30: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:24:36.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:36.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:36.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:36.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:36.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:36.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:36.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:36.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:36.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:36.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:36.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:36.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:36.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:36.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:36.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:36.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:36.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:36.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:36.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:36.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:36.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:36.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:36.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:36.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:36.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:36.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:36.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:36.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:36.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:36.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:36.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:36.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:36.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:36.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:36.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:36.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:36.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:36.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:36.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:36.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:36.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:36.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:36.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:36.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:36.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:36.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:36.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:36.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:36.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:36.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:36.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:36.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:36.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:36.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:36.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:36.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:36.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:36.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:36.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:36.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:36.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:36.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:36.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:36.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:36.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:36.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:36.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:36.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:36.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:36.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:36.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:36.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:36.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:36.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:36.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:36.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:36.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:36.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:36.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:36.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:36.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:36.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:36.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:36.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:36.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:36.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:36.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:36.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:36.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:36.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:36.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:36.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:36.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:36.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:36.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:36.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:36.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:36.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:36.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:36.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:36.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:36.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:36.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:36.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:36.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:36.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:36.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:36.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:36.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:36.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:36.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:36.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:36.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:36.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:36.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: pgmap v30: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:24:36.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:36.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:36.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:36.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:36.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:36.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:36.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:36.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:36.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:36.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:36.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:36.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:36.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:36.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:36.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:36.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:36.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:36.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:36.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:36.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:36.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:36.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:36.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:36.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:36.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:36.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:36.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:36.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:36.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:36.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:36.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:36.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:36.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:36.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:36.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:36.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:36.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:36.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:36.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:36.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:36.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:36.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:36.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:36.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:36.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:36.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:36.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:36.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:36.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:36.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:36.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:36.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:36.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:36.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:36.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:36.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:36.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:36.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:36.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:36.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:36.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:36.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:36.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:36.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:36.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:36.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:36.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:36.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:36.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:36.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:36 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:37.422 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:37.422 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:37.422 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:37.422 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:37.422 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:37.423 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:37.423 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:37.423 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:37.423 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:37.423 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:37.423 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:37.423 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:37.423 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:37.423 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:37.423 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:37.423 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:37.423 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:37.423 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:37.424 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:37.424 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:37.424 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:37.424 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:37.424 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:37.424 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:37.424 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:37.424 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:37.424 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:37.424 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:37.424 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:37.425 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:37.425 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:37.425 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:37.425 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:37.425 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:37.425 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:37.425 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:37.425 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:37.425 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:37.425 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:37.425 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:37.425 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:37.425 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:37.425 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:37.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:37.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:37.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:37.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:37.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:37.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:37.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:37.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:37.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:37.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:37.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:37.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:37.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:37.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:37.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:37.427 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:37.427 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:37.427 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:37.427 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:37.427 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:37.427 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:37.427 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:37.427 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:37.427 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:37.427 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:37.427 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:37.428 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:37.428 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:37.428 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:37.428 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:37.428 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:37.428 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:37.428 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:37.428 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:37.428 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:37.428 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:37.428 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:37.428 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:37.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:37.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:37.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:37.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:37.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:37.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:37.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:37.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:37.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:37.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:37.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:37.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:37.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:37.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:37.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:37.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:37.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:37.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:37.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:37.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:37 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:37.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:37 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:37.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:37 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:37.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:37 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:37.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:37 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:37.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:37 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:37.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:37 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:37.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:37 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:37.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:37 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:37.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:37 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:37.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:37 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:37.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:37 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:37.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:37 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:37.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:37 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:37.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:37 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:37.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:37 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:37.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:37 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:37.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:37 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:37.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:37 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:37.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:37 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:37.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:37 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:37.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:37 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:37.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:37 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:37.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:37 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:37.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:37 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:37.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:37 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:37.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:37 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:37.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:37 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:37.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:37 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:37.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:37 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:37.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:37 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:37.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:37 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:37.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:37 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:37.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:37 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:37.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:37 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:37.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:37 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:37.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:37 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:37.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:37 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:37.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:37 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:37.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:37 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:37.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:37 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:37.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:37 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:37.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:37 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:37.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:37 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:37.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:37 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:37.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:37 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:37.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:37 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:37.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:37 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:37.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:37 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:37.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:37 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:38.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: pgmap v31: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:24:38.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:38.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:38.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:38.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:38.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:38.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:38.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:38.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:38.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:38.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:38.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:38.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:38.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:38.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:38.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:38.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:38.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:38.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:38.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:38.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:38.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:38.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:38.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:38.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:38.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:38.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:38.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:38.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:38.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: pgmap v31: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:24:38.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:38.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:38.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:38.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:38.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:38.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:38.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:38.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:38.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:38.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:38.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:38.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:38.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:38.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:38.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:38.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:38.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:38.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:38.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:38.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:38.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:38.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:38.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:38.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:38.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:38.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:38.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:38.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:38.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:38.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:38.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:38.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:38.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:38.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:38.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:38.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:38.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:38.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:38.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:38.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:38.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:38.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:38.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:38.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:38.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:38.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:38.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:38.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:38.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:38.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:38.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:38.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:38.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:38.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:38.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:38.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:38.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:38.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:38.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:38.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:38.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:38.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:38.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:38.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:38.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:38.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:38.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:38.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:38.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:38.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:38.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:38.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:38.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:38.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:38.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:38.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:38.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:38.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:38.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:38.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:38.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:38.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:38.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:38.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:38.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:38.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:38.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:38.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:38.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:38.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:38.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: pgmap v31: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:24:38.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:38.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:38.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:38.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:38.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:38.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:38.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:38.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:38.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:38.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:38.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:38.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:38.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:38.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:38.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:38.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:38.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:38.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:38.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:38.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:38.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:38.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:38.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:38.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:38.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:38.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:38.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:38.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:38.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:38.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:38.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:38.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:38.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:38.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:38.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:38.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:38.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:38.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:38.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:38.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:38.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:38.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:38.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:38.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:38.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:38.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:38.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:38.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:38.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:38.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:38.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:38.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:38.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:38.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:38.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:38.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:38.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:38.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:38.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:39.108 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:24:38 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:24:38.659Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"smithi062:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"172.21.15.139:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-24T11:24:39.447 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:39.447 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:39.447 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:39.447 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:39.447 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:39.447 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:39.447 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:39.448 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:39.448 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:39.448 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:39.448 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:39.448 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:39.448 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:39.448 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:39.448 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:39.448 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:39.448 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:39.448 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:39.448 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:39.448 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:39.448 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:39.448 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:39.449 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:39.449 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:39.449 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:39.449 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:39.449 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:39.449 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:39.449 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:39.449 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:39.449 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:39.450 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:39.450 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:39.450 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:39.450 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:39.450 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:39.450 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:39.450 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:39.450 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:39.450 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:39.450 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:39.450 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:39.450 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:39.450 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:39.450 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:39.451 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:39.451 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:39.451 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:39.451 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:39.451 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:39.451 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:39.451 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:39.451 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:39.451 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:39.451 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:39.451 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:39.451 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:39.451 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:39.451 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:39.451 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:39.452 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:39.452 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:39.452 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:39.452 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:39.452 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:39.452 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:39.452 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:39.452 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:39.452 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:39.452 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:39.452 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:39.452 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:39.452 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:39.452 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:39.452 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:39.452 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:39.453 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:39.453 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:39.453 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:39.453 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:39.453 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:39.453 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:39.453 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:39.453 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:39.453 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:39.453 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:39.453 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:39.453 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:39.453 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:39.454 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:39.454 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:39.454 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:39.454 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:39.454 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:39.454 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:39.454 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:39.454 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:39.454 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:39.454 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:39.454 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:39.454 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:39.454 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:39.454 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:39.455 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:39.455 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:39.455 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:39.455 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:39.455 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:39.455 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:39.455 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:39.455 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:39.455 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:39.455 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:39.455 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:39.455 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:39.455 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:39.455 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:39.455 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:39.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:39.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:39.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:39.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:39.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:39.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:39.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:39.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:39.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:39.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:39.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:39.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:39.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:39.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:39.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:39.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:39.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:39.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:39.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:39.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:39.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:39.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:39.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:39.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:39.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:39.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:39.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:39.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:39.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:39.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:39.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:39.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:39.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:39.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:39.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:39.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:39.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:39.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:39.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:39.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:39.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:39.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:39.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:39.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:39.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:39.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:39.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:39.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:39.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:39.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:39.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:39.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:39.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:39.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:39.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:39.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:39.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:39.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:39.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:39 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:40.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:40.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[90553]: pgmap v32: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:24:40.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:40.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:40.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:40.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:40.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:40.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:40.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:40.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:40.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:40.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:40.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:40.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:40.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:40.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:40.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:40.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:40.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:40.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:40.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:40.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:40.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:40.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:40.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:40.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:40.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:40.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:40.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:40.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:40.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:40.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:40.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:40.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:40.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:40.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:40.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:40.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:40.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:40.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:40.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:40.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:40.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:40.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:40.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:40.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:40.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:40.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:40.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:40.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:40.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:40.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[98643]: pgmap v32: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:24:40.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:40.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:40.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:40.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:40.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:40.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:40.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:40.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:40.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:40.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:40.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:40.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:40.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:40.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:40.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:40.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:40.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:40.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:40.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:40.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:40.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:40.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:40.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:40.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:40.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:40.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:40.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:40.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:40.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:40.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:40.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:40.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:40.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:40.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:40.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:40.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:40.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:40.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:40.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:40.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:40.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:40.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:40.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:40.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:40.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:40.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:40.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:40.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:40.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:40 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:40.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:40 smithi139 ceph-mon[25719]: pgmap v32: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:24:40.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:40 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:40.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:40 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:40.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:40 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:40.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:40 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:40.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:40 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:40.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:40 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:40.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:40 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:40.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:40 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:40.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:40 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:40.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:40 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:40.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:40 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:40.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:40 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:40.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:40 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:40.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:40 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:40.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:40 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:40.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:40 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:40.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:40 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:40.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:40 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:40.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:40 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:40.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:40 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:40.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:40 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:40.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:40 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:40.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:40 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:40.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:40 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:40.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:40 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:40.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:40 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:40.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:40 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:40.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:40 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:40.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:40 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:40.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:40 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:40.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:40 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:40.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:40 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:40.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:40 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:40.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:40 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:40.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:40 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:40.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:40 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:40.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:40 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:40.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:40 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:40.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:40 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:40.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:40 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:40.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:40 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:40.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:40 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:40.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:40 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:40.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:40 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:40.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:40 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:40.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:40 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:40.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:40 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:40.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:40 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:41.123 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:24:40 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:24:40] "GET /metrics HTTP/1.1" 200 38330 "" "Prometheus/2.33.4" 2024-03-24T11:24:41.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:41.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:41.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:41.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:41.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:41.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:41.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:41.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:41.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:41.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:41.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:41.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:41.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:41.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:41.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:41.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:41.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:41.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:41.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:41.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:41.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:41.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:41.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:41.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:41.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:41.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:41.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:41.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:41.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:41.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:41.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:41.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:41.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:41.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:41.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:41.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:41.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:41.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:41.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:41.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:41.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:41.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:41.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:41.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:41.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:41.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:41.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:41.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:41.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:41.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:41.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:41.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:41.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:41.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:41.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:41.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:41.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:41.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:41.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:41.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:41.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:41.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:41.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:41.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:41.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:41.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:41.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:41.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:41.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:41.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:41.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:41.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:41.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:41.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:41.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:41.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:41.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:41.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:41.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:41.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:41.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:41.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:41.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:41.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:41.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:41.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:41.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:41.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:41.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:41.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:41.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:41.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:41.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:41.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:41.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:41.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:41.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:41.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:41.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:41.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:41.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:41.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:41.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:41.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:41.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:41.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:41.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:41.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:41.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:41.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:41.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:41.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:41.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:41.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:41.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:41.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:41.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:41.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:41.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:24:41.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:41.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:41.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:41.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:24:41.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:41.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:41.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:41.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:41.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:41.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:41.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:41.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:41.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:41.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:41.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:41.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:41.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:41.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:41.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:41.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:41.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:41.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:41.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:41.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:41.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:41.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:41.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:41.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:41.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:41.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:41.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:41.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:41.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:41.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:41.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:41.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:41.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:41.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:41.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:41.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:41.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:41.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:41.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:41.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:41.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:41.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:41.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:41.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:41.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:41.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:41.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:41.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:41.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:41.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:41.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:41.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:41.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:41.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:41.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:41.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:41.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:41.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:41.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:41.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:41.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:41.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:24:41.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:41 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:42.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:42.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: pgmap v33: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:24:42.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:42.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:42.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:42.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:42.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:42.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:42.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:42.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:42.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:42.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:42.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:42.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:42.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:42.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:42.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:42.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:42.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:42.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:42.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:42.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:42.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:42.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:42.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:42.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:42.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:42.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:42.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:42.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:42.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: pgmap v33: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:24:42.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:42.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:42.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:42.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:42.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:42.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:42.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:42.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:42.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:42.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:42.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:42.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:42.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:42.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:42.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:42.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:42.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:42.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:42.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:42.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:42.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:42.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:42.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:42.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:42.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:42.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:42.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:42.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:42.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:42.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:42.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:42.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:42.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:42.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:42.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:42.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:42.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:42.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:42.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:42.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:42.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:42.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:42.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:42.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:42.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:42.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:42.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:42.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:42.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:42.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:42.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:42.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:42.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:42.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:42.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:42.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:42.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:42.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:42.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:42.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:42.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:42.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:42.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:42.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:42.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:42.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:42.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:42.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:42.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:42.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:42.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:42.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:42.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:42.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:42.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:42.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:42.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:42.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:42.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:42.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:42.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:42.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:42.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:42.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:42.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:42.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:42.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:42.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:42.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:42.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:42.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:42.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:42.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:42.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:42.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:42.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:42.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:42.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:42.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:42.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:42.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:42.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:42.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:42.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:42.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:42.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:42.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:42.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:42.490 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:42.490 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:42.490 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:42.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:42.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: pgmap v33: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:24:42.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:42.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:42.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:42.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:42.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:42.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:42.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:42.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:42.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:42.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:42.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:42.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:42.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:42.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:42.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:42.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:42.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:42.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:42.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:42.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:42.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:42.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:42.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:42.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:42.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:42.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:42.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:42.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:42.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:42.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:42.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:42.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:42.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:42.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:42.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:42.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:42.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:42.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:42.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:42.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:42.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:42.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:42.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:42.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:42.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:42.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:42.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:42.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:42.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:42.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:42.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:42.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:42.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:42.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:42.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:42.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:42.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:42.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:42.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:42.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:42.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:42.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:42.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:42.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:42.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:42.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:42.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:42.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:42.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:42 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:43.221 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:43.222 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:24:43.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:43.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:24:43.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:43.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:43.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:43.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:43.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:43.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:43.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:43.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:43.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:43.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:43.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:43.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:43.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:43.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:43.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:43.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:43.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:43.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:43.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:43.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:43.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:43.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:43.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:43.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:43.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:43.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:43.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:43.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:43.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:43.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:43.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:43.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:43.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:43.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:43.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:43.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:43.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:43.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:43.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:43.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:43.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:43.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:43.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:43.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:43.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:43.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:43.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:43.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:43.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:43.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:43.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:43.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:43.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:43.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:43.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:43.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:43.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:43.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:43.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:43.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:43.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:43.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:43.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:43.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:43.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:43.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:43.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:43.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:43.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:43.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:43.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:43.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:43.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:43.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:43.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:43.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:43.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:43.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:43.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:43.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:43.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:43.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:43.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:43.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:43.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:43.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:43.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:43.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:43.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:43.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:43.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:43.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:43.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:43.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:43.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:43.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:43.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:43.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:43.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:43.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:43.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:43.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:43.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:43.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:43.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:43.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:43.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:43.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:43.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:43.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:43.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:43.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:43.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:43.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:43.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:43.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:43.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:43.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:43.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:43.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:43.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:43.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:24:43.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:43.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:43.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:43.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:43.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:43.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:43.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:43.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:43.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:43.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:43.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:43.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:43.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:43.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:43.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:43.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:43.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:43.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:43.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:43.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:43.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:43.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:43.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:43.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:43.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:43.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:43.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:43.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:43.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:43.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:43.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:43.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:43.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:43.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:43.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:43.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:43.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:43.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:43.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:43.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:43.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:43.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:43.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:43.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:43.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:43.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:43.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:43.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:43.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:43.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:43.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:43.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:43.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:43.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:43.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:43.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:43.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:43.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:43.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:43.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:43 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:44.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: pgmap v34: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:24:44.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:44.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:44.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:44.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:44.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:44.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:44.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:44.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:44.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:44.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:44.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:44.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:44.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:44.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:44.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:44.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:44.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:44.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:44.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:44.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:44.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:44.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:44.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:44.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:44.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:44.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:44.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:44.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:44.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: pgmap v34: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:24:44.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:44.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:44.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:44.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:44.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:44.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:44.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:44.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:44.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:44.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:44.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:44.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:44.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:44.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:44.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:44.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:44.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:44.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:44.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:44.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:44.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:44.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:44.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:44.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:44.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:44.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:44.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:44.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:44.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:44.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:44.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:44.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:44.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:44.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:44.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:44.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:44.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:44.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:44.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:44.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:44.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:44.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:44.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:44.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:44.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:44.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:44.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:44.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:44.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:44.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:44.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:44.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:44.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:44.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:44.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:44.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:44.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:44.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:44.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:44.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:44.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:44.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:44.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:44.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:44.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:44.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:44.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:44.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:44.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:44.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:44.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:44.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:44.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:44.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:44.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:44.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:44.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:44.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:44.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:44.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:44.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:44.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:44.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:44.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:44.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:44.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:44.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:44.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:44.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:44.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:44.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:44.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:44.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: pgmap v34: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:24:44.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:44.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:44.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:44.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:44.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:44.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:44.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:44.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:44.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:44.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:44.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:44.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:44.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:44.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:44.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:44.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:44.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:44.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:44.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:44.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:44.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:44.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:44.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:44.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:44.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:44.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:44.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:44.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:44.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:44.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:44.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:44.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:44.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:44.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:44.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:44.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:44.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:44.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:44.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:44.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:44.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:44.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:44.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:44.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:44.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:44.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:44.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:44.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:44.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:44.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:44.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:44.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:44.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:44.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:44.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:44.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:44.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:44.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:44.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:44.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:44 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:45.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:45.554 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:45.555 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:45.555 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:45.555 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:45.555 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:45.555 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:45.555 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:45.555 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:45.555 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:45.555 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:45.555 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:45.555 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:45.555 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:45.555 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:45.556 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:45.556 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:45.556 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:45.556 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:45.556 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:45.556 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:45.556 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:45.556 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:45.556 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:45.556 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:45.556 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:45.556 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:45.556 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:45.556 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:45.556 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:45.557 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:45.557 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:45.557 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:45.557 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:45.557 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:45.557 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:45.557 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:45.557 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:45.557 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:45.557 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:45.558 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:45.558 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:45.558 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:45.558 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:45.558 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:45.558 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:45.558 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:45.558 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:45.558 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:45.558 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:45.558 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:45.558 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:45.558 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:45.558 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:45.558 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:45.559 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:45.559 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:45.559 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:45.559 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:45.559 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:45.559 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:45.559 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:45.559 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:45.559 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:45.560 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:45.560 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:45.560 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:45.560 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:45.560 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:45.560 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:45.560 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:45.560 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:45.560 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:45.560 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:45.560 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:45.560 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:45.560 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:45.560 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:45.560 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:45.561 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:45.561 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:45.561 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:45.561 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:45.561 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:45.561 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:45.561 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:45.561 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:45.561 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:45.561 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:45.561 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:45.561 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:45.561 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:45.561 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:45.562 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:45.562 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:45.562 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:45.562 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:45.562 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:45.562 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:45.562 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:45.562 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:45.562 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:45.562 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:45.563 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:45.563 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:45.563 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:45.563 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:45.563 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:45.563 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:45.563 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:45.563 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:45.563 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:45.563 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:45.563 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:45.563 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:45.563 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:45.565 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:45.565 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:45.565 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:45.565 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:45.565 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:45.565 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:45.565 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:45.565 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:45.565 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:45.565 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:45.566 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:45.566 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:45.566 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:45.566 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:45.566 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:45.566 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:45.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:45.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:45.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:45.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:45.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:45.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:45.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:45.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:45.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:45.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:45.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:45.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:45.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:45.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:45.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:45.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:45.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:45.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:45.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:45.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:45.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:45.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:45.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:45.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:45.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:45.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:45.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:45.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:45.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:45.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:45.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:45.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:45.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:45.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:45.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:45.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:45.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:45.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:45.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:45.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:45.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:45.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:45.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:45.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:45.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:45.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:45.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:45.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:45.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:45.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:45.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:45.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:45.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:45.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:45.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:45.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:45.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:45.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:45.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:45.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:45.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:45.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:45.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:45.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:45.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:45.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:45 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:46.411 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: pgmap v35: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:24:46.411 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:46.411 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:46.412 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:46.412 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:46.412 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:46.412 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:46.412 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:46.412 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:46.412 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:46.412 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:46.412 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:46.412 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:46.412 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:46.412 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:46.412 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:46.412 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:46.413 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:46.413 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:46.413 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:46.413 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:46.413 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:46.413 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:46.413 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:46.413 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:46.413 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:46.413 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:46.413 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:46.413 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:46.414 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: pgmap v35: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:24:46.414 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:46.414 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:46.414 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:46.414 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:46.414 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:46.414 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:46.414 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:46.414 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:46.414 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:46.414 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:46.414 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:46.414 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:46.415 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:46.415 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:46.415 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:46.415 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:46.415 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:46.415 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:46.415 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:46.415 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:46.415 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:46.415 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:46.415 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:46.415 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:46.415 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:46.415 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:46.415 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:46.416 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:46.416 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:46.416 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:46.416 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:46.416 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:46.416 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:46.416 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:46.416 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:46.417 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:46.417 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:46.417 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:46.417 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:46.417 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:46.417 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:46.417 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:46.417 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:46.417 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:46.417 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:46.417 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:46.417 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:46.417 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:46.417 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:46.417 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:46.418 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:46.418 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:46.418 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:46.418 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:46.418 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:46.418 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:46.418 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:46.418 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:46.418 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:46.418 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:46.418 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:46.418 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:46.418 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:46.418 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:46.418 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:46.419 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:46.419 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:46.419 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:46.419 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:46.419 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:46.419 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:46.419 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:46.419 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:46.419 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:46.419 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:46.420 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:46.420 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:46.420 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:46.420 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:46.420 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:46.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:46.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:46.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:46.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:46.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:46.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:46.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:46.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:46.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:46.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:46.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:46.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:46.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:46.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:46.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:46.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:46.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:46.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:46.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:46.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:46.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:46.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:46.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:46.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:46.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:46.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:46.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:46.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:46.423 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:46.423 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:46.423 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:46.423 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:46.423 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:46.423 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:46.423 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:46.423 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:46.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: pgmap v35: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:24:46.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:46.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:46.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:46.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:46.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:46.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:46.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:46.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:46.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:46.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:46.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:46.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:46.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:46.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:46.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:46.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:46.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:46.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:46.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:46.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:46.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:46.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:46.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:46.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:46.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:46.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:46.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:46.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:46.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:46.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:46.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:46.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:46.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:46.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:46.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:46.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:46.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:46.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:46.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:46.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:46.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:46.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:46.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:46.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:46.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:46.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:46.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:46.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:46.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:46.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:46.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:46.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:46.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:46.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:46.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:46.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:46.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:46.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:46.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:46.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:46.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:46.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:46.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:46.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:46.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:46.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:46.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:46.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:46.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:46.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:46.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:46.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:46 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:46.879 INFO:teuthology.orchestra.run.smithi062.stdout:true 2024-03-24T11:24:47.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:47.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:47.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:47.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:47.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:47.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:47.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:47.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:47.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:47.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:47.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:47.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:47.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:47.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:47.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:47.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:47.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:47.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:47.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:47.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:47.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:47.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:47.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:47.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:47.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:47.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:47.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:47.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:47.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:47.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:47.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:47.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:47.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:47.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:47.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:47.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:47.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:47.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:47.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:47.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:47.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:47.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:47.238 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:47.238 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:47.238 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: from='client.64102 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:24:47.238 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:47.238 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:47.238 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:47.238 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:47.238 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:47.238 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:47.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:47.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:47.239 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:47.239 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:47.239 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:47.239 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:47.240 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:47.240 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:47.240 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:47.240 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:47.240 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:47.240 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:47.240 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:47.240 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:47.240 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:47.240 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:47.240 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:47.240 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:47.240 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:47.240 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:47.241 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:47.241 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:47.241 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:47.241 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:47.241 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:47.241 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:47.241 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:47.241 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:47.241 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:47.241 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:47.241 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:47.241 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:47.241 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:47.241 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:47.241 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:47.242 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:47.242 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:47.242 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:47.242 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:47.242 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:47.242 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:47.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:47.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:47.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:47.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:47.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:47.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:47.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:47.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:47.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:47.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:47.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:47.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:47.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:47.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:47.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:47.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:47.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:47.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:47.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:47.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:47.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:47.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:47.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:47.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:47.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:47.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:47.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:47.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:47.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:47.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:47.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:47.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:47.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:47.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:47.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:47.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:47.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:47.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:47.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:47.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:47.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:47.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:47.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:47.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:47.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: from='client.64102 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:24:47.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:47.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:47.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:47.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:47.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:47.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:47.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:47.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:47.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:47.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:47 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:47.629 INFO:teuthology.orchestra.run.smithi062.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T11:24:47.629 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi062 smithi062 running 18s ago 14m - - 2024-03-24T11:24:47.629 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi139 smithi139 running 16s ago 14m - - 2024-03-24T11:24:47.629 INFO:teuthology.orchestra.run.smithi062.stdout:alertmanager.a smithi062 *:9093,9094 running (4m) 18s ago 8m 25.1M - 0.23.0 ba2b418f427c 818298d58140 2024-03-24T11:24:47.629 INFO:teuthology.orchestra.run.smithi062.stdout:grafana.a smithi139 *:3000 running (3m) 16s ago 8m 53.8M - 8.3.5 dad864ee21e9 b48167e7171e 2024-03-24T11:24:47.629 INFO:teuthology.orchestra.run.smithi062.stdout:iscsi.foo.smithi062.dggsvk smithi062 running (2m) 18s ago 7m 56.0M - 3.5 e1d6a67b021e 0a98806ec839 2024-03-24T11:24:47.629 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.x smithi139 *:8443,9283,8765 running (92s) 16s ago 13m 448M - 19.0.0-1667-gdb0330b1 4f325f8440e9 d94987b11387 2024-03-24T11:24:47.629 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.y smithi062 *:8443,9283,8765 running (2m) 18s ago 15m 505M - 19.0.0-1667-gdb0330b1 4f325f8440e9 ca1a49028d65 2024-03-24T11:24:47.629 INFO:teuthology.orchestra.run.smithi062.stdout:mon.a smithi062 running (48s) 18s ago 15m 42.5M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 f7afe6e28c88 2024-03-24T11:24:47.629 INFO:teuthology.orchestra.run.smithi062.stdout:mon.b smithi139 running (13m) 16s ago 13m 59.3M 2048M 17.2.0 e1d6a67b021e 8945cdb4e6aa 2024-03-24T11:24:47.630 INFO:teuthology.orchestra.run.smithi062.stdout:mon.c smithi062 starting - - - 2048M 2024-03-24T11:24:47.630 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.a smithi062 *:9100 running (4m) 18s ago 9m 17.5M - 1.3.1 1dbe0e931976 79d5a02a999f 2024-03-24T11:24:47.630 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.b smithi139 *:9100 running (3m) 16s ago 9m 15.5M - 1.3.1 1dbe0e931976 8f9f4608e465 2024-03-24T11:24:47.630 INFO:teuthology.orchestra.run.smithi062.stdout:osd.0 smithi062 running (13m) 18s ago 13m 54.0M 4096M 17.2.0 e1d6a67b021e 8040b30d7170 2024-03-24T11:24:47.630 INFO:teuthology.orchestra.run.smithi062.stdout:osd.1 smithi062 running (12m) 18s ago 12m 55.5M 4096M 17.2.0 e1d6a67b021e e62d22e5dc29 2024-03-24T11:24:47.630 INFO:teuthology.orchestra.run.smithi062.stdout:osd.2 smithi062 running (12m) 18s ago 12m 49.1M 4096M 17.2.0 e1d6a67b021e 714f550103a7 2024-03-24T11:24:47.630 INFO:teuthology.orchestra.run.smithi062.stdout:osd.3 smithi062 running (11m) 18s ago 11m 53.6M 4096M 17.2.0 e1d6a67b021e 1d2868bd5195 2024-03-24T11:24:47.630 INFO:teuthology.orchestra.run.smithi062.stdout:osd.4 smithi139 running (11m) 16s ago 11m 57.8M 4096M 17.2.0 e1d6a67b021e 7568bfa9e74d 2024-03-24T11:24:47.630 INFO:teuthology.orchestra.run.smithi062.stdout:osd.5 smithi139 running (11m) 16s ago 11m 51.0M 4096M 17.2.0 e1d6a67b021e dd0abd599a14 2024-03-24T11:24:47.630 INFO:teuthology.orchestra.run.smithi062.stdout:osd.6 smithi139 running (10m) 16s ago 10m 50.6M 4096M 17.2.0 e1d6a67b021e a4b1289aca69 2024-03-24T11:24:47.630 INFO:teuthology.orchestra.run.smithi062.stdout:osd.7 smithi139 running (10m) 16s ago 10m 53.4M 4096M 17.2.0 e1d6a67b021e bc7fadbc1384 2024-03-24T11:24:47.630 INFO:teuthology.orchestra.run.smithi062.stdout:prometheus.a smithi139 *:9095 running (2m) 16s ago 9m 60.1M - 2.33.4 514e6a882f6e 2ed47f810d33 2024-03-24T11:24:47.630 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi062.sacpds smithi062 *:8000 running (8m) 18s ago 8m 73.0M - 17.2.0 e1d6a67b021e 89f4ac308e98 2024-03-24T11:24:47.630 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi139.yljesh smithi139 *:8000 running (7m) 16s ago 7m 71.7M - 17.2.0 e1d6a67b021e e189bdc12cae 2024-03-24T11:24:47.631 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi062.xvpxnk smithi062 *:80 running (7m) 18s ago 7m 72.8M - 17.2.0 e1d6a67b021e d1aa74adab44 2024-03-24T11:24:47.631 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi139.pbtrmp smithi139 *:80 running (7m) 16s ago 7m 75.2M - 17.2.0 e1d6a67b021e 7f8e336755d7 2024-03-24T11:24:47.631 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:47.631 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:47.631 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:47.631 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:47.632 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:47.632 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:47.632 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:47.632 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: from='client.64102 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:24:47.632 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:47.632 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:47.632 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:47.632 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:47.632 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:47.632 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:47.632 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:47.632 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:47.632 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:47.632 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:48.097 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:24:48.097 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": { 2024-03-24T11:24:48.097 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 1, 2024-03-24T11:24:48.097 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-24T11:24:48.097 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:24:48.097 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": { 2024-03-24T11:24:48.097 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-24T11:24:48.097 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:24:48.097 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": { 2024-03-24T11:24:48.097 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 8 2024-03-24T11:24:48.097 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:24:48.097 INFO:teuthology.orchestra.run.smithi062.stdout: "rgw": { 2024-03-24T11:24:48.098 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4 2024-03-24T11:24:48.098 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:24:48.098 INFO:teuthology.orchestra.run.smithi062.stdout: "overall": { 2024-03-24T11:24:48.098 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 13, 2024-03-24T11:24:48.098 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 4 2024-03-24T11:24:48.098 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-24T11:24:48.098 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:24:48.446 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:24:48.446 INFO:teuthology.orchestra.run.smithi062.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69", 2024-03-24T11:24:48.446 INFO:teuthology.orchestra.run.smithi062.stdout: "in_progress": true, 2024-03-24T11:24:48.446 INFO:teuthology.orchestra.run.smithi062.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T11:24:48.446 INFO:teuthology.orchestra.run.smithi062.stdout: "services_complete": [ 2024-03-24T11:24:48.446 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr" 2024-03-24T11:24:48.447 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-03-24T11:24:48.447 INFO:teuthology.orchestra.run.smithi062.stdout: "progress": "4/23 daemons upgraded", 2024-03-24T11:24:48.447 INFO:teuthology.orchestra.run.smithi062.stdout: "message": "Currently upgrading mon daemons", 2024-03-24T11:24:48.447 INFO:teuthology.orchestra.run.smithi062.stdout: "is_paused": false 2024-03-24T11:24:48.447 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:24:48.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:48 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:48.688 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:48 smithi139 ceph-mon[25719]: pgmap v36: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:24:48.689 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:48 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:48.689 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:48 smithi139 ceph-mon[25719]: from='client.64108 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:24:48.689 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:48.689 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:48.689 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:48.689 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:48.689 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:48.689 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:48 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:48.689 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:48.689 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:48.689 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:48.689 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:48.690 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:48.690 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:48 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:48.690 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:48.690 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:48.690 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:48.690 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:48.690 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:48.690 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:48 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:48.690 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:48 smithi139 ceph-mon[25719]: from='client.64114 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:24:48.690 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:48.690 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:48.690 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:48.690 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:48.691 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:48.691 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:48 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:48.691 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:48.691 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:48.691 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:48.691 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:48.691 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:48.691 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:48 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:48.691 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:48.691 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:48.691 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:48.691 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:48.691 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:48.691 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:48 smithi139 ceph-mon[25719]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:48.691 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:48.692 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:48 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/508551402' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:48.692 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:48.692 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:48.692 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:48.692 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:48.692 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-24T11:24:48.692 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:48 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mon ok-to-stop", "ids": ["b"]}]: dispatch 2024-03-24T11:24:48.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:48.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[90553]: pgmap v36: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:24:48.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:48.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[90553]: from='client.64108 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:24:48.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:48.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:48.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:48.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:48.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:48.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:48.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:48.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:48.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:48.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:48.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:48.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:48.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:48.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:48.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:48.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:48.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:48.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:48.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[90553]: from='client.64114 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:24:48.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:48.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:48.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:48.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:48.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:48.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:48.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:48.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:48.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:48.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:48.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:48.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:48.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:48.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:48.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:48.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:48.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:48.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:48.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:48.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[90553]: from='client.? 172.21.15.62:0/508551402' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:48.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:48.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:48.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:48.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:48.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-24T11:24:48.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mon ok-to-stop", "ids": ["b"]}]: dispatch 2024-03-24T11:24:48.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:48.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[98643]: pgmap v36: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:24:48.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:48.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[98643]: from='client.64108 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:24:48.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:48.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:48.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:48.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:48.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:48.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:48.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:48.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:48.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:48.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:48.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:48.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:48.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:48.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:48.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:48.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:48.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:48.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:48.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[98643]: from='client.64114 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:24:48.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:48.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:48.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:48.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:48.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:48.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:48.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:48.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:48.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:48.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:48.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:48.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:48.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:48.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:48.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:48.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:48.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:48.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:48.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:48.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[98643]: from='client.? 172.21.15.62:0/508551402' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:48.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:48.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:48.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:24:48.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:24:48.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-24T11:24:48.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mon ok-to-stop", "ids": ["b"]}]: dispatch 2024-03-24T11:24:48.859 INFO:teuthology.orchestra.run.smithi062.stdout:HEALTH_OK 2024-03-24T11:24:48.976 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:24:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:24:48.658Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"smithi062:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"172.21.15.139:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-24T11:24:49.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:49 smithi062 ceph-mon[90553]: Upgrade: It appears safe to stop mon.b 2024-03-24T11:24:49.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:49 smithi062 ceph-mon[90553]: from='client.44160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:24:49.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:49 smithi062 ceph-mon[90553]: from='client.? 172.21.15.62:0/530705512' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:24:49.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:49 smithi062 ceph-mon[98643]: Upgrade: It appears safe to stop mon.b 2024-03-24T11:24:49.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:49 smithi062 ceph-mon[98643]: from='client.44160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:24:49.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:49 smithi062 ceph-mon[98643]: from='client.? 172.21.15.62:0/530705512' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:24:49.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:49 smithi139 ceph-mon[25719]: Upgrade: It appears safe to stop mon.b 2024-03-24T11:24:49.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:49 smithi139 ceph-mon[25719]: from='client.44160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:24:49.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:49 smithi139 ceph-mon[25719]: from='client.? 172.21.15.62:0/530705512' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:24:50.826 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:50 smithi139 ceph-mon[25719]: pgmap v37: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:24:50.826 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:50 smithi139 ceph-mon[25719]: Upgrade: Updating mon.b 2024-03-24T11:24:50.826 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:50 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:50.826 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:50 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T11:24:50.826 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:50 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T11:24:50.826 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:50 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:50.826 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:50 smithi139 ceph-mon[25719]: Deploying daemon mon.b on smithi139 2024-03-24T11:24:50.826 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:50 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:50.827 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:50 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:50.827 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:50 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:50.976 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:24:50 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:24:50] "GET /metrics HTTP/1.1" 200 38331 "" "Prometheus/2.33.4" 2024-03-24T11:24:50.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:50 smithi062 ceph-mon[90553]: pgmap v37: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:24:50.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:50 smithi062 ceph-mon[90553]: Upgrade: Updating mon.b 2024-03-24T11:24:50.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:50 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:50.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:50 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T11:24:50.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:50 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T11:24:50.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:50 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:50.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:50 smithi062 ceph-mon[90553]: Deploying daemon mon.b on smithi139 2024-03-24T11:24:50.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:50 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:50.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:50 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:50.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:50 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:50.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:50 smithi062 ceph-mon[98643]: pgmap v37: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:24:50.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:50 smithi062 ceph-mon[98643]: Upgrade: Updating mon.b 2024-03-24T11:24:50.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:50 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:50.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:50 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T11:24:50.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:50 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T11:24:50.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:50 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:50.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:50 smithi062 ceph-mon[98643]: Deploying daemon mon.b on smithi139 2024-03-24T11:24:50.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:50 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:50.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:50 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:50.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:50 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:51.653 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:51 smithi139 systemd[1]: Stopping Ceph mon.b for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:24:52.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:52 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:52.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:52 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:52.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:52 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:52.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:52 smithi062 ceph-mon[90553]: pgmap v38: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:24:52.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:52 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:52.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:52 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:52.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:52 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:52.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:52 smithi062 ceph-mon[98643]: pgmap v38: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:24:52.293 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:52 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:52.293 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:52 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:52.293 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:52 smithi139 ceph-mon[25719]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:52.293 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:52 smithi139 ceph-mon[25719]: pgmap v38: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:24:52.607 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:52 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-b[25697]: 2024-03-24T11:24:52.292+0000 7f2779279700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T11:24:52.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:52 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-b[25697]: 2024-03-24T11:24:52.292+0000 7f2779279700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-03-24T11:24:53.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:52 smithi139 podman[91729]: 2024-03-24 11:24:52.856701597 +0000 UTC m=+0.665040517 container died 8945cdb4e6aa0e97119eb73f8d53fb090369f42a3fd04dbbb49c2472a4879024 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-b, io.openshift.expose-services=, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, io.buildah.version=1.19.8, CEPH_POINT_RELEASE=-17.2.0, GIT_BRANCH=HEAD, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, io.openshift.tags=base centos centos-stream, GIT_CLEAN=True, architecture=x86_64, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vendor=Red Hat, Inc., com.redhat.license_terms=https://centos.org/legal/licensing-policy/, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, name=centos-stream, release=754, vcs-type=git, RELEASE=HEAD, io.k8s.display-name=CentOS Stream 8, maintainer=Guillaume Abrioux , url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, version=8, com.redhat.component=centos-stream-container, ceph=True, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, build-date=2022-05-03T08:36:31.336870, distribution-scope=public, GIT_REPO=https://github.com/ceph/ceph-container.git) 2024-03-24T11:24:53.405 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:53 smithi139 podman[91729]: 2024-03-24 11:24:53.22041442 +0000 UTC m=+1.028753343 container cleanup 8945cdb4e6aa0e97119eb73f8d53fb090369f42a3fd04dbbb49c2472a4879024 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-b, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.k8s.display-name=CentOS Stream 8, io.buildah.version=1.19.8, GIT_REPO=https://github.com/ceph/ceph-container.git, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, build-date=2022-05-03T08:36:31.336870, release=754, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, com.redhat.component=centos-stream-container, CEPH_POINT_RELEASE=-17.2.0, GIT_BRANCH=HEAD, io.openshift.tags=base centos centos-stream, ceph=True, vcs-type=git, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, io.openshift.expose-services=, maintainer=Guillaume Abrioux , name=centos-stream, GIT_CLEAN=True, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, vendor=Red Hat, Inc., architecture=x86_64, version=8, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, RELEASE=HEAD, distribution-scope=public) 2024-03-24T11:24:53.405 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:53 smithi139 bash[91729]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-b 2024-03-24T11:24:53.732 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:53 smithi139 podman[91746]: 2024-03-24 11:24:53.405368949 +0000 UTC m=+0.879679797 container remove 8945cdb4e6aa0e97119eb73f8d53fb090369f42a3fd04dbbb49c2472a4879024 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-b, version=8, io.openshift.expose-services=, GIT_CLEAN=True, distribution-scope=public, io.k8s.display-name=CentOS Stream 8, name=centos-stream, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, CEPH_POINT_RELEASE=-17.2.0, GIT_REPO=https://github.com/ceph/ceph-container.git, architecture=x86_64, io.openshift.tags=base centos centos-stream, vendor=Red Hat, Inc., RELEASE=HEAD, com.redhat.component=centos-stream-container, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, maintainer=Guillaume Abrioux , description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, vcs-type=git, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, release=754, ceph=True, build-date=2022-05-03T08:36:31.336870, GIT_BRANCH=HEAD, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, io.buildah.version=1.19.8, com.redhat.license_terms=https://centos.org/legal/licensing-policy/) 2024-03-24T11:24:53.732 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:53 smithi139 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mon.b.service: Deactivated successfully. 2024-03-24T11:24:53.732 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:53 smithi139 systemd[1]: Stopped Ceph mon.b for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:24:53.732 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:53 smithi139 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mon.b.service: Consumed 13.721s CPU time. 2024-03-24T11:24:54.046 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:53 smithi139 systemd[1]: Starting Ceph mon.b for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:24:54.297 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 podman[91876]: 2024-03-24 11:24:53.948211529 +0000 UTC m=+0.017852756 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:24:54.297 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 podman[91876]: 2024-03-24 11:24:54.06464309 +0000 UTC m=+0.134284305 container create ca357beea4dd13c6c6cc4a856ed836fe6e96b3e966014469650898c4b3682bfe (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240307, RELEASE=squid-db0330b, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T11:24:54.297 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 podman[91876]: 2024-03-24 11:24:54.294348793 +0000 UTC m=+0.363990014 container init ca357beea4dd13c6c6cc4a856ed836fe6e96b3e966014469650898c4b3682bfe (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-24T11:24:54.297 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 podman[91876]: 2024-03-24 11:24:54.2975461 +0000 UTC m=+0.367187315 container start ca357beea4dd13c6c6cc4a856ed836fe6e96b3e966014469650898c4b3682bfe (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, ceph=True) 2024-03-24T11:24:54.600 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: set uid:gid to 167:167 (ceph:ceph) 2024-03-24T11:24:54.601 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev), process ceph-mon, pid 2 2024-03-24T11:24:54.601 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: pidfile_write: ignore empty --pid-file 2024-03-24T11:24:54.601 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: load: jerasure load: lrc 2024-03-24T11:24:54.601 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: RocksDB version: 7.9.2 2024-03-24T11:24:54.601 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Git sha 0 2024-03-24T11:24:54.601 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Compile date 2024-03-16 10:36:11 2024-03-24T11:24:54.601 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: DB SUMMARY 2024-03-24T11:24:54.601 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: DB Session ID: C4K1HYU51H8C37RHZBLJ 2024-03-24T11:24:54.601 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: CURRENT file: CURRENT 2024-03-24T11:24:54.601 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: IDENTITY file: IDENTITY 2024-03-24T11:24:54.601 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: MANIFEST file: MANIFEST-000009 size: 2228 Bytes 2024-03-24T11:24:54.601 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 1, files: 000045.sst 2024-03-24T11:24:54.602 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000043.log size: 1743419 ; 2024-03-24T11:24:54.602 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.error_if_exists: 0 2024-03-24T11:24:54.602 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.create_if_missing: 0 2024-03-24T11:24:54.602 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.paranoid_checks: 1 2024-03-24T11:24:54.602 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-24T11:24:54.602 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-24T11:24:54.602 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-24T11:24:54.602 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.env: 0x55df0a6f0ae0 2024-03-24T11:24:54.602 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.fs: PosixFileSystem 2024-03-24T11:24:54.602 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.info_log: 0x55df0c7638c0 2024-03-24T11:24:54.602 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.max_file_opening_threads: 16 2024-03-24T11:24:54.602 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.statistics: (nil) 2024-03-24T11:24:54.603 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.use_fsync: 0 2024-03-24T11:24:54.603 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.max_log_file_size: 0 2024-03-24T11:24:54.603 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-24T11:24:54.603 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-24T11:24:54.603 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.keep_log_file_num: 1000 2024-03-24T11:24:54.603 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.recycle_log_file_num: 0 2024-03-24T11:24:54.603 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.allow_fallocate: 1 2024-03-24T11:24:54.603 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.allow_mmap_reads: 0 2024-03-24T11:24:54.603 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.allow_mmap_writes: 0 2024-03-24T11:24:54.603 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.use_direct_reads: 0 2024-03-24T11:24:54.603 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-24T11:24:54.603 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.create_missing_column_families: 0 2024-03-24T11:24:54.604 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.db_log_dir: 2024-03-24T11:24:54.604 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.wal_dir: 2024-03-24T11:24:54.604 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-24T11:24:54.604 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-24T11:24:54.604 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-24T11:24:54.604 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-24T11:24:54.604 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-24T11:24:54.604 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-24T11:24:54.604 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.advise_random_on_open: 1 2024-03-24T11:24:54.604 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.db_write_buffer_size: 0 2024-03-24T11:24:54.604 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.write_buffer_manager: 0x55df0c736dc0 2024-03-24T11:24:54.605 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-24T11:24:54.605 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-24T11:24:54.605 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-24T11:24:54.605 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.rate_limiter: (nil) 2024-03-24T11:24:54.605 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-24T11:24:54.605 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.wal_recovery_mode: 2 2024-03-24T11:24:54.605 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.enable_thread_tracking: 0 2024-03-24T11:24:54.605 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.enable_pipelined_write: 0 2024-03-24T11:24:54.605 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.unordered_write: 0 2024-03-24T11:24:54.605 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-24T11:24:54.605 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-24T11:24:54.605 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-24T11:24:54.606 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-24T11:24:54.606 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.row_cache: None 2024-03-24T11:24:54.606 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.wal_filter: None 2024-03-24T11:24:54.606 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-24T11:24:54.606 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.allow_ingest_behind: 0 2024-03-24T11:24:54.606 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.two_write_queues: 0 2024-03-24T11:24:54.606 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.manual_wal_flush: 0 2024-03-24T11:24:54.606 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.wal_compression: 0 2024-03-24T11:24:54.606 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.atomic_flush: 0 2024-03-24T11:24:54.606 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-24T11:24:54.607 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-24T11:24:54.607 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-24T11:24:54.607 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.log_readahead_size: 0 2024-03-24T11:24:54.607 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-24T11:24:54.607 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.best_efforts_recovery: 0 2024-03-24T11:24:54.607 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-24T11:24:54.607 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-24T11:24:54.607 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.allow_data_in_errors: 0 2024-03-24T11:24:54.607 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.db_host_id: __hostname__ 2024-03-24T11:24:54.607 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-24T11:24:54.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.max_background_jobs: 2 2024-03-24T11:24:54.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.max_background_compactions: -1 2024-03-24T11:24:54.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.max_subcompactions: 1 2024-03-24T11:24:54.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-24T11:24:54.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-24T11:24:54.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-24T11:24:54.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.max_total_wal_size: 0 2024-03-24T11:24:54.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-24T11:24:54.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-24T11:24:54.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-24T11:24:54.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-24T11:24:54.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.max_open_files: -1 2024-03-24T11:24:54.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.bytes_per_sync: 0 2024-03-24T11:24:54.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-24T11:24:54.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-24T11:24:54.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.compaction_readahead_size: 0 2024-03-24T11:24:54.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.max_background_flushes: -1 2024-03-24T11:24:54.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Compression algorithms supported: 2024-03-24T11:24:54.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-24T11:24:54.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: kZSTD supported: 0 2024-03-24T11:24:54.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: kXpressCompression supported: 0 2024-03-24T11:24:54.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: kLZ4HCCompression supported: 1 2024-03-24T11:24:54.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: kZlibCompression supported: 1 2024-03-24T11:24:54.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: kSnappyCompression supported: 1 2024-03-24T11:24:54.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: kLZ4Compression supported: 1 2024-03-24T11:24:54.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: kBZip2Compression supported: 0 2024-03-24T11:24:54.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-24T11:24:54.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-24T11:24:54.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000009 2024-03-24T11:24:54.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-24T11:24:54.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-24T11:24:54.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.merge_operator: 2024-03-24T11:24:54.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.compaction_filter: None 2024-03-24T11:24:54.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.compaction_filter_factory: None 2024-03-24T11:24:54.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.sst_partitioner_factory: None 2024-03-24T11:24:54.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-24T11:24:54.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-24T11:24:54.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55df0c7639c0) 2024-03-24T11:24:54.611 INFO:journalctl@ceph.mon.b.smithi139.stdout: cache_index_and_filter_blocks: 1 2024-03-24T11:24:54.611 INFO:journalctl@ceph.mon.b.smithi139.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-24T11:24:54.611 INFO:journalctl@ceph.mon.b.smithi139.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-24T11:24:54.611 INFO:journalctl@ceph.mon.b.smithi139.stdout: pin_top_level_index_and_filter: 1 2024-03-24T11:24:54.611 INFO:journalctl@ceph.mon.b.smithi139.stdout: index_type: 0 2024-03-24T11:24:54.611 INFO:journalctl@ceph.mon.b.smithi139.stdout: data_block_index_type: 0 2024-03-24T11:24:54.611 INFO:journalctl@ceph.mon.b.smithi139.stdout: index_shortening: 1 2024-03-24T11:24:54.611 INFO:journalctl@ceph.mon.b.smithi139.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-24T11:24:54.611 INFO:journalctl@ceph.mon.b.smithi139.stdout: checksum: 4 2024-03-24T11:24:54.611 INFO:journalctl@ceph.mon.b.smithi139.stdout: no_block_cache: 0 2024-03-24T11:24:54.612 INFO:journalctl@ceph.mon.b.smithi139.stdout: block_cache: 0x55df0c767090 2024-03-24T11:24:54.612 INFO:journalctl@ceph.mon.b.smithi139.stdout: block_cache_name: BinnedLRUCache 2024-03-24T11:24:54.612 INFO:journalctl@ceph.mon.b.smithi139.stdout: block_cache_options: 2024-03-24T11:24:54.612 INFO:journalctl@ceph.mon.b.smithi139.stdout: capacity : 536870912 2024-03-24T11:24:54.612 INFO:journalctl@ceph.mon.b.smithi139.stdout: num_shard_bits : 4 2024-03-24T11:24:54.612 INFO:journalctl@ceph.mon.b.smithi139.stdout: strict_capacity_limit : 0 2024-03-24T11:24:54.612 INFO:journalctl@ceph.mon.b.smithi139.stdout: high_pri_pool_ratio: 0.000 2024-03-24T11:24:54.612 INFO:journalctl@ceph.mon.b.smithi139.stdout: block_cache_compressed: (nil) 2024-03-24T11:24:54.612 INFO:journalctl@ceph.mon.b.smithi139.stdout: persistent_cache: (nil) 2024-03-24T11:24:54.612 INFO:journalctl@ceph.mon.b.smithi139.stdout: block_size: 4096 2024-03-24T11:24:54.612 INFO:journalctl@ceph.mon.b.smithi139.stdout: block_size_deviation: 10 2024-03-24T11:24:54.613 INFO:journalctl@ceph.mon.b.smithi139.stdout: block_restart_interval: 16 2024-03-24T11:24:54.613 INFO:journalctl@ceph.mon.b.smithi139.stdout: index_block_restart_interval: 1 2024-03-24T11:24:54.613 INFO:journalctl@ceph.mon.b.smithi139.stdout: metadata_block_size: 4096 2024-03-24T11:24:54.613 INFO:journalctl@ceph.mon.b.smithi139.stdout: partition_filters: 0 2024-03-24T11:24:54.613 INFO:journalctl@ceph.mon.b.smithi139.stdout: use_delta_encoding: 1 2024-03-24T11:24:54.613 INFO:journalctl@ceph.mon.b.smithi139.stdout: filter_policy: bloomfilter 2024-03-24T11:24:54.613 INFO:journalctl@ceph.mon.b.smithi139.stdout: whole_key_filtering: 1 2024-03-24T11:24:54.613 INFO:journalctl@ceph.mon.b.smithi139.stdout: verify_compression: 0 2024-03-24T11:24:54.613 INFO:journalctl@ceph.mon.b.smithi139.stdout: read_amp_bytes_per_bit: 0 2024-03-24T11:24:54.613 INFO:journalctl@ceph.mon.b.smithi139.stdout: format_version: 5 2024-03-24T11:24:54.613 INFO:journalctl@ceph.mon.b.smithi139.stdout: enable_index_compression: 1 2024-03-24T11:24:54.613 INFO:journalctl@ceph.mon.b.smithi139.stdout: block_align: 0 2024-03-24T11:24:54.613 INFO:journalctl@ceph.mon.b.smithi139.stdout: max_auto_readahead_size: 262144 2024-03-24T11:24:54.614 INFO:journalctl@ceph.mon.b.smithi139.stdout: prepopulate_block_cache: 0 2024-03-24T11:24:54.614 INFO:journalctl@ceph.mon.b.smithi139.stdout: initial_auto_readahead_size: 8192 2024-03-24T11:24:54.614 INFO:journalctl@ceph.mon.b.smithi139.stdout: num_file_reads_for_auto_readahead: 2 2024-03-24T11:24:54.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.write_buffer_size: 33554432 2024-03-24T11:24:54.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.max_write_buffer_number: 2 2024-03-24T11:24:54.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.compression: NoCompression 2024-03-24T11:24:54.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.bottommost_compression: Disabled 2024-03-24T11:24:54.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.prefix_extractor: nullptr 2024-03-24T11:24:54.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-24T11:24:54.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.num_levels: 7 2024-03-24T11:24:54.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-24T11:24:54.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-24T11:24:54.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-24T11:24:54.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-24T11:24:54.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-24T11:24:54.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-24T11:24:54.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-24T11:24:54.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-24T11:24:54.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-24T11:24:54.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-24T11:24:54.616 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-24T11:24:54.616 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-24T11:24:54.616 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-24T11:24:54.616 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.compression_opts.level: 32767 2024-03-24T11:24:54.616 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.compression_opts.strategy: 0 2024-03-24T11:24:54.616 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-24T11:24:54.616 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-24T11:24:54.616 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-24T11:24:54.616 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-24T11:24:54.616 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.compression_opts.enabled: false 2024-03-24T11:24:54.616 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-24T11:24:54.616 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-24T11:24:54.616 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-24T11:24:54.617 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-24T11:24:54.617 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.target_file_size_base: 67108864 2024-03-24T11:24:54.617 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-24T11:24:54.617 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-24T11:24:54.617 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-24T11:24:54.617 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-24T11:24:54.617 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-24T11:24:54.617 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-24T11:24:54.617 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-24T11:24:54.617 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-24T11:24:54.617 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-24T11:24:54.617 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-24T11:24:54.618 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-24T11:24:54.618 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-24T11:24:54.618 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-24T11:24:54.618 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-24T11:24:54.618 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.arena_block_size: 1048576 2024-03-24T11:24:54.618 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-24T11:24:54.618 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-24T11:24:54.618 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.disable_auto_compactions: 0 2024-03-24T11:24:54.618 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-24T11:24:54.618 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-24T11:24:54.618 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-24T11:24:54.619 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-24T11:24:54.620 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-24T11:24:54.620 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-24T11:24:54.620 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-24T11:24:54.620 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-24T11:24:54.620 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-24T11:24:54.620 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-24T11:24:54.620 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.table_properties_collectors: 2024-03-24T11:24:54.620 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.inplace_update_support: 0 2024-03-24T11:24:54.620 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-24T11:24:54.620 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-24T11:24:54.621 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-24T11:24:54.621 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-24T11:24:54.621 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.bloom_locality: 0 2024-03-24T11:24:54.621 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.max_successive_merges: 0 2024-03-24T11:24:54.621 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-24T11:24:54.621 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.paranoid_file_checks: 0 2024-03-24T11:24:54.621 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.force_consistency_checks: 1 2024-03-24T11:24:54.621 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.report_bg_io_stats: 0 2024-03-24T11:24:54.621 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.ttl: 2592000 2024-03-24T11:24:54.621 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-24T11:24:54.621 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-24T11:24:54.621 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-24T11:24:54.621 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.enable_blob_files: false 2024-03-24T11:24:54.622 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.min_blob_size: 0 2024-03-24T11:24:54.622 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.blob_file_size: 268435456 2024-03-24T11:24:54.622 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-24T11:24:54.622 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-24T11:24:54.622 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-24T11:24:54.622 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-24T11:24:54.622 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-24T11:24:54.622 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.blob_file_starting_level: 0 2024-03-24T11:24:54.622 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-24T11:24:54.622 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 45.sst 2024-03-24T11:24:54.622 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-24T11:24:54.622 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 47, last_sequence is 29412, log_number is 43,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-24T11:24:54.622 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 43 2024-03-24T11:24:54.622 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: a40d8d9b-295b-492f-98de-0c0801c36337 2024-03-24T11:24:54.623 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711279494327091, "job": 1, "event": "recovery_started", "wal_files": [43]} 2024-03-24T11:24:54.623 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #43 mode 2 2024-03-24T11:24:54.623 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711279494334791, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 48, "file_size": 1115946, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 29417, "largest_seqno": 31283, "table_properties": {"data_size": 1109638, "index_size": 2669, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 2629, "raw_key_size": 25709, "raw_average_key_size": 24, "raw_value_size": 1092767, "raw_average_value_size": 1047, "num_data_blocks": 105, "num_entries": 1043, "num_filter_entries": 1043, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711279494, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a40d8d9b-295b-492f-98de-0c0801c36337", "db_session_id": "C4K1HYU51H8C37RHZBLJ", "orig_file_number": 48, "seqno_to_time_mapping": "N/A"}} 2024-03-24T11:24:54.623 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711279494334896, "job": 1, "event": "recovery_finished"} 2024-03-24T11:24:54.623 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: [db/version_set.cc:5047] Creating manifest 50 2024-03-24T11:24:54.623 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-24T11:24:54.623 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000043.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-24T11:24:54.623 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55df0c858000 2024-03-24T11:24:54.623 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: DB pointer 0x55df0c842000 2024-03-24T11:24:54.623 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-24T11:24:54.623 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-24T11:24:54.623 INFO:journalctl@ceph.mon.b.smithi139.stdout: ** DB Stats ** 2024-03-24T11:24:54.623 INFO:journalctl@ceph.mon.b.smithi139.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-24T11:24:54.624 INFO:journalctl@ceph.mon.b.smithi139.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-24T11:24:54.624 INFO:journalctl@ceph.mon.b.smithi139.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-24T11:24:54.624 INFO:journalctl@ceph.mon.b.smithi139.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-24T11:24:54.624 INFO:journalctl@ceph.mon.b.smithi139.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-24T11:24:54.624 INFO:journalctl@ceph.mon.b.smithi139.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-24T11:24:54.624 INFO:journalctl@ceph.mon.b.smithi139.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-24T11:24:54.624 INFO:journalctl@ceph.mon.b.smithi139.stdout: 2024-03-24T11:24:54.624 INFO:journalctl@ceph.mon.b.smithi139.stdout: ** Compaction Stats [default] ** 2024-03-24T11:24:54.624 INFO:journalctl@ceph.mon.b.smithi139.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-24T11:24:54.625 INFO:journalctl@ceph.mon.b.smithi139.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-24T11:24:54.625 INFO:journalctl@ceph.mon.b.smithi139.stdout: L0 1/0 1.06 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 400.5 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-03-24T11:24:54.625 INFO:journalctl@ceph.mon.b.smithi139.stdout: L6 1/0 11.38 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-03-24T11:24:54.625 INFO:journalctl@ceph.mon.b.smithi139.stdout: Sum 2/0 12.44 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 400.5 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-03-24T11:24:54.625 INFO:journalctl@ceph.mon.b.smithi139.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 400.5 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-03-24T11:24:54.625 INFO:journalctl@ceph.mon.b.smithi139.stdout: 2024-03-24T11:24:54.625 INFO:journalctl@ceph.mon.b.smithi139.stdout: ** Compaction Stats [default] ** 2024-03-24T11:24:54.625 INFO:journalctl@ceph.mon.b.smithi139.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-24T11:24:54.625 INFO:journalctl@ceph.mon.b.smithi139.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-24T11:24:54.625 INFO:journalctl@ceph.mon.b.smithi139.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 400.5 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-03-24T11:24:54.626 INFO:journalctl@ceph.mon.b.smithi139.stdout: 2024-03-24T11:24:54.626 INFO:journalctl@ceph.mon.b.smithi139.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-24T11:24:54.626 INFO:journalctl@ceph.mon.b.smithi139.stdout: 2024-03-24T11:24:54.626 INFO:journalctl@ceph.mon.b.smithi139.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-24T11:24:54.626 INFO:journalctl@ceph.mon.b.smithi139.stdout: Flush(GB): cumulative 0.001, interval 0.001 2024-03-24T11:24:54.626 INFO:journalctl@ceph.mon.b.smithi139.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-24T11:24:54.626 INFO:journalctl@ceph.mon.b.smithi139.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-24T11:24:54.626 INFO:journalctl@ceph.mon.b.smithi139.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-24T11:24:54.626 INFO:journalctl@ceph.mon.b.smithi139.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-24T11:24:54.626 INFO:journalctl@ceph.mon.b.smithi139.stdout: Cumulative compaction: 0.00 GB write, 88.85 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-24T11:24:54.626 INFO:journalctl@ceph.mon.b.smithi139.stdout: Interval compaction: 0.00 GB write, 88.85 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-24T11:24:54.627 INFO:journalctl@ceph.mon.b.smithi139.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-24T11:24:54.627 INFO:journalctl@ceph.mon.b.smithi139.stdout: Block cache BinnedLRUCache@0x55df0c767090#2 capacity: 512.00 MB usage: 5.61 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-03-24T11:24:54.627 INFO:journalctl@ceph.mon.b.smithi139.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,2.78 KB,0.000530481%) IndexBlock(1,2.83 KB,0.000539422%) Misc(1,0.00 KB,0%) 2024-03-24T11:24:54.627 INFO:journalctl@ceph.mon.b.smithi139.stdout: 2024-03-24T11:24:54.627 INFO:journalctl@ceph.mon.b.smithi139.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-24T11:24:54.627 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: starting mon.b rank 2 at public addrs [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] at bind addrs [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid ebb89568-e9ce-11ee-95cd-87774f69a715 2024-03-24T11:24:54.627 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: mon.b@-1(???) e3 preinit fsid ebb89568-e9ce-11ee-95cd-87774f69a715 2024-03-24T11:24:54.627 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: mon.b@-1(???).mds e1 new map 2024-03-24T11:24:54.627 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: mon.b@-1(???).mds e1 print_map 2024-03-24T11:24:54.627 INFO:journalctl@ceph.mon.b.smithi139.stdout: e1 2024-03-24T11:24:54.627 INFO:journalctl@ceph.mon.b.smithi139.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T11:24:54.627 INFO:journalctl@ceph.mon.b.smithi139.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-24T11:24:54.627 INFO:journalctl@ceph.mon.b.smithi139.stdout: legacy client fscid: -1 2024-03-24T11:24:54.627 INFO:journalctl@ceph.mon.b.smithi139.stdout: 2024-03-24T11:24:54.628 INFO:journalctl@ceph.mon.b.smithi139.stdout: No filesystems configured 2024-03-24T11:24:54.628 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: mon.b@-1(???).osd e108 crush map has features 3314933000854323200, adjusting msgr requires 2024-03-24T11:24:54.628 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: mon.b@-1(???).osd e108 crush map has features 432629239337189376, adjusting msgr requires 2024-03-24T11:24:54.628 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: mon.b@-1(???).osd e108 crush map has features 432629239337189376, adjusting msgr requires 2024-03-24T11:24:54.628 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: mon.b@-1(???).osd e108 crush map has features 432629239337189376, adjusting msgr requires 2024-03-24T11:24:54.628 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 ceph-mon[91894]: mon.b@-1(???).paxosservice(auth 1..28) refresh upgraded, format 0 -> 3 2024-03-24T11:24:54.628 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 bash[91876]: ca357beea4dd13c6c6cc4a856ed836fe6e96b3e966014469650898c4b3682bfe 2024-03-24T11:24:54.628 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:54 smithi139 systemd[1]: Started Ceph mon.b for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:24:55.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:55 smithi139 ceph-mon[91894]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:24:55.976 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:55 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:24:55.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:55 smithi062 ceph-mon[98643]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:24:56.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:56 smithi139 ceph-mon[91894]: pgmap v40: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:24:56.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T11:24:56.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:56 smithi139 ceph-mon[91894]: mon.a calling monitor election 2024-03-24T11:24:56.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:56 smithi139 ceph-mon[91894]: mon.c calling monitor election 2024-03-24T11:24:56.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:56 smithi139 ceph-mon[91894]: mon.b calling monitor election 2024-03-24T11:24:56.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T11:24:56.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:24:56.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:56 smithi139 ceph-mon[91894]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-24T11:24:56.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:56 smithi139 ceph-mon[91894]: monmap e4: 3 mons at {a=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0],b=[v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0],c=[v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-24T11:24:56.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:56 smithi139 ceph-mon[91894]: fsmap 2024-03-24T11:24:56.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:56 smithi139 ceph-mon[91894]: osdmap e108: 8 total, 8 up, 8 in 2024-03-24T11:24:56.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:56 smithi139 ceph-mon[91894]: mgrmap e44: y(active, since 74s), standbys: x 2024-03-24T11:24:56.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:56 smithi139 ceph-mon[91894]: overall HEALTH_OK 2024-03-24T11:24:56.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:56.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:56.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:56.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:56.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:56.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:56 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:56.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:56 smithi139 ceph-mon[91894]: Reconfiguring mon.a (monmap changed)... 2024-03-24T11:24:56.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T11:24:56.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T11:24:56.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:56.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:56 smithi139 ceph-mon[91894]: Reconfiguring daemon mon.a on smithi062 2024-03-24T11:24:56.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:24:56.967 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:56 smithi062 ceph-mon[90553]: pgmap v40: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:24:56.967 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T11:24:56.967 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:56 smithi062 ceph-mon[90553]: mon.a calling monitor election 2024-03-24T11:24:56.967 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:56 smithi062 ceph-mon[90553]: mon.c calling monitor election 2024-03-24T11:24:56.967 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:56 smithi062 ceph-mon[90553]: mon.b calling monitor election 2024-03-24T11:24:56.967 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T11:24:56.967 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:24:56.967 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:56 smithi062 ceph-mon[90553]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-24T11:24:56.967 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:56 smithi062 ceph-mon[90553]: monmap e4: 3 mons at {a=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0],b=[v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0],c=[v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-24T11:24:56.967 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:56 smithi062 ceph-mon[90553]: fsmap 2024-03-24T11:24:56.968 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:56 smithi062 ceph-mon[90553]: osdmap e108: 8 total, 8 up, 8 in 2024-03-24T11:24:56.968 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:56 smithi062 ceph-mon[90553]: mgrmap e44: y(active, since 74s), standbys: x 2024-03-24T11:24:56.968 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:56 smithi062 ceph-mon[90553]: overall HEALTH_OK 2024-03-24T11:24:56.968 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:56.968 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:56.968 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:56.968 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:56.968 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:56.968 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:56 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:56.968 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:56 smithi062 ceph-mon[90553]: Reconfiguring mon.a (monmap changed)... 2024-03-24T11:24:56.968 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T11:24:56.968 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T11:24:56.968 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:56.968 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:56 smithi062 ceph-mon[90553]: Reconfiguring daemon mon.a on smithi062 2024-03-24T11:24:56.969 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:24:56.969 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:56 smithi062 ceph-mon[98643]: pgmap v40: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:24:56.969 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T11:24:56.969 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:56 smithi062 ceph-mon[98643]: mon.a calling monitor election 2024-03-24T11:24:56.969 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:56 smithi062 ceph-mon[98643]: mon.c calling monitor election 2024-03-24T11:24:56.969 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:56 smithi062 ceph-mon[98643]: mon.b calling monitor election 2024-03-24T11:24:56.969 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T11:24:56.969 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T11:24:56.969 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:56 smithi062 ceph-mon[98643]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-24T11:24:56.969 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:56 smithi062 ceph-mon[98643]: monmap e4: 3 mons at {a=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0],b=[v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0],c=[v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-24T11:24:56.969 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:56 smithi062 ceph-mon[98643]: fsmap 2024-03-24T11:24:56.970 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:56 smithi062 ceph-mon[98643]: osdmap e108: 8 total, 8 up, 8 in 2024-03-24T11:24:56.970 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:56 smithi062 ceph-mon[98643]: mgrmap e44: y(active, since 74s), standbys: x 2024-03-24T11:24:56.970 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:56 smithi062 ceph-mon[98643]: overall HEALTH_OK 2024-03-24T11:24:56.970 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:56.970 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:56.970 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:56.970 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:24:56.970 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:56.970 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:56 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:24:56.970 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:56 smithi062 ceph-mon[98643]: Reconfiguring mon.a (monmap changed)... 2024-03-24T11:24:56.970 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T11:24:56.970 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T11:24:56.970 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:56.970 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:56 smithi062 ceph-mon[98643]: Reconfiguring daemon mon.a on smithi062 2024-03-24T11:24:56.970 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:24:58.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:58 smithi062 ceph-mon[90553]: pgmap v41: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:24:58.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:58.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:58.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:58 smithi062 ceph-mon[90553]: Reconfiguring mgr.y (monmap changed)... 2024-03-24T11:24:58.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T11:24:58.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T11:24:58.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:58.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:24:58 smithi062 ceph-mon[90553]: Reconfiguring daemon mgr.y on smithi062 2024-03-24T11:24:58.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:58 smithi062 ceph-mon[98643]: pgmap v41: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:24:58.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:58.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:58.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:58 smithi062 ceph-mon[98643]: Reconfiguring mgr.y (monmap changed)... 2024-03-24T11:24:58.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T11:24:58.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T11:24:58.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:58.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:24:58 smithi062 ceph-mon[98643]: Reconfiguring daemon mgr.y on smithi062 2024-03-24T11:24:58.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:58 smithi139 ceph-mon[91894]: pgmap v41: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:24:58.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:58.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:24:58.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:58 smithi139 ceph-mon[91894]: Reconfiguring mgr.y (monmap changed)... 2024-03-24T11:24:58.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T11:24:58.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T11:24:58.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:24:58.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:24:58 smithi139 ceph-mon[91894]: Reconfiguring daemon mgr.y on smithi062 2024-03-24T11:24:59.108 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:24:58 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:24:58.659Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"smithi062:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"172.21.15.139:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-24T11:24:59.650 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:24:59 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:24:59.346+0000 7f4e4aad7700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-03-24T11:25:00.282 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:00.282 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:00.282 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:00 smithi062 ceph-mon[90553]: pgmap v42: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:25:00.282 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:00 smithi062 ceph-mon[90553]: Reconfiguring mon.c (monmap changed)... 2024-03-24T11:25:00.282 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T11:25:00.282 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T11:25:00.282 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:00.282 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:00 smithi062 ceph-mon[90553]: Reconfiguring daemon mon.c on smithi062 2024-03-24T11:25:00.282 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:00.283 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:00.283 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:00 smithi062 ceph-mon[98643]: pgmap v42: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:25:00.283 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:00 smithi062 ceph-mon[98643]: Reconfiguring mon.c (monmap changed)... 2024-03-24T11:25:00.283 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T11:25:00.283 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T11:25:00.283 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:00.283 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:00 smithi062 ceph-mon[98643]: Reconfiguring daemon mon.c on smithi062 2024-03-24T11:25:00.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:00.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:00.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:00 smithi139 ceph-mon[91894]: pgmap v42: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:25:00.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:00 smithi139 ceph-mon[91894]: Reconfiguring mon.c (monmap changed)... 2024-03-24T11:25:00.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T11:25:00.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T11:25:00.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:00.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:00 smithi139 ceph-mon[91894]: Reconfiguring daemon mon.c on smithi062 2024-03-24T11:25:00.976 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:25:00 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:25:00] "GET /metrics HTTP/1.1" 200 38269 "" "Prometheus/2.33.4" 2024-03-24T11:25:01.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:01.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:01.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:01 smithi062 ceph-mon[90553]: Reconfiguring osd.0 (monmap changed)... 2024-03-24T11:25:01.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-24T11:25:01.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:01.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:01 smithi062 ceph-mon[90553]: Reconfiguring daemon osd.0 on smithi062 2024-03-24T11:25:01.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:01.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:01.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:01 smithi062 ceph-mon[98643]: Reconfiguring osd.0 (monmap changed)... 2024-03-24T11:25:01.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-24T11:25:01.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:01.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:01 smithi062 ceph-mon[98643]: Reconfiguring daemon osd.0 on smithi062 2024-03-24T11:25:02.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:02.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:02.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:01 smithi139 ceph-mon[91894]: Reconfiguring osd.0 (monmap changed)... 2024-03-24T11:25:02.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-24T11:25:02.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:02.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:01 smithi139 ceph-mon[91894]: Reconfiguring daemon osd.0 on smithi062 2024-03-24T11:25:02.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:02 smithi062 ceph-mon[90553]: pgmap v43: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:25:02.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:02.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:02.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-24T11:25:02.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:02.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:02 smithi062 ceph-mon[98643]: pgmap v43: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:25:02.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:02.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:02.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-24T11:25:02.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:03.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:02 smithi139 ceph-mon[91894]: pgmap v43: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:25:03.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:03.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:03.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-24T11:25:03.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:03.848 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:03 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:25:03.849 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:03 smithi062 ceph-mon[90553]: Reconfiguring osd.1 (monmap changed)... 2024-03-24T11:25:03.849 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:03 smithi062 ceph-mon[90553]: Reconfiguring daemon osd.1 on smithi062 2024-03-24T11:25:03.849 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:03 smithi062 ceph-mon[98643]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:25:03.849 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:03 smithi062 ceph-mon[98643]: Reconfiguring osd.1 (monmap changed)... 2024-03-24T11:25:03.849 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:03 smithi062 ceph-mon[98643]: Reconfiguring daemon osd.1 on smithi062 2024-03-24T11:25:04.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:03 smithi139 ceph-mon[91894]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:25:04.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:03 smithi139 ceph-mon[91894]: Reconfiguring osd.1 (monmap changed)... 2024-03-24T11:25:04.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:03 smithi139 ceph-mon[91894]: Reconfiguring daemon osd.1 on smithi062 2024-03-24T11:25:05.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:04 smithi062 ceph-mon[90553]: pgmap v44: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:25:05.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:05.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:05.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:04 smithi062 ceph-mon[90553]: Reconfiguring osd.2 (monmap changed)... 2024-03-24T11:25:05.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-24T11:25:05.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:05.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:04 smithi062 ceph-mon[90553]: Reconfiguring daemon osd.2 on smithi062 2024-03-24T11:25:05.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:04 smithi062 ceph-mon[98643]: pgmap v44: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:25:05.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:05.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:05.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:04 smithi062 ceph-mon[98643]: Reconfiguring osd.2 (monmap changed)... 2024-03-24T11:25:05.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-24T11:25:05.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:05.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:04 smithi062 ceph-mon[98643]: Reconfiguring daemon osd.2 on smithi062 2024-03-24T11:25:05.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:04 smithi139 ceph-mon[91894]: pgmap v44: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:25:05.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:05.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:05.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:04 smithi139 ceph-mon[91894]: Reconfiguring osd.2 (monmap changed)... 2024-03-24T11:25:05.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-24T11:25:05.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:05.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:04 smithi139 ceph-mon[91894]: Reconfiguring daemon osd.2 on smithi062 2024-03-24T11:25:06.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:06 smithi139 ceph-mon[91894]: pgmap v45: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:25:06.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:06.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:06.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:06 smithi139 ceph-mon[91894]: Reconfiguring osd.3 (monmap changed)... 2024-03-24T11:25:06.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-24T11:25:06.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:06.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:06 smithi139 ceph-mon[91894]: Reconfiguring daemon osd.3 on smithi062 2024-03-24T11:25:06.913 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:06 smithi062 ceph-mon[98643]: pgmap v45: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:25:06.913 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:06.914 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:06.914 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:06 smithi062 ceph-mon[98643]: Reconfiguring osd.3 (monmap changed)... 2024-03-24T11:25:06.914 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-24T11:25:06.914 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:06.914 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:06 smithi062 ceph-mon[98643]: Reconfiguring daemon osd.3 on smithi062 2024-03-24T11:25:06.914 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:06 smithi062 ceph-mon[90553]: pgmap v45: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:25:06.914 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:06.914 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:06.914 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:06 smithi062 ceph-mon[90553]: Reconfiguring osd.3 (monmap changed)... 2024-03-24T11:25:06.914 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-24T11:25:06.915 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:06.915 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:06 smithi062 ceph-mon[90553]: Reconfiguring daemon osd.3 on smithi062 2024-03-24T11:25:08.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:08 smithi062 ceph-mon[98643]: pgmap v46: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:25:08.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:08.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:08.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:08 smithi062 ceph-mon[98643]: Reconfiguring rgw.foo.smithi062.sacpds (monmap changed)... 2024-03-24T11:25:08.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi062.sacpds", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-03-24T11:25:08.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:08.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:08 smithi062 ceph-mon[98643]: Reconfiguring daemon rgw.foo.smithi062.sacpds on smithi062 2024-03-24T11:25:08.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:08 smithi062 ceph-mon[90553]: pgmap v46: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:25:08.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:08.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:08.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:08 smithi062 ceph-mon[90553]: Reconfiguring rgw.foo.smithi062.sacpds (monmap changed)... 2024-03-24T11:25:08.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi062.sacpds", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-03-24T11:25:08.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:08.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:08 smithi062 ceph-mon[90553]: Reconfiguring daemon rgw.foo.smithi062.sacpds on smithi062 2024-03-24T11:25:08.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:08 smithi139 ceph-mon[91894]: pgmap v46: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:25:08.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:08.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:08.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:08 smithi139 ceph-mon[91894]: Reconfiguring rgw.foo.smithi062.sacpds (monmap changed)... 2024-03-24T11:25:08.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi062.sacpds", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-03-24T11:25:08.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:08.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:08 smithi139 ceph-mon[91894]: Reconfiguring daemon rgw.foo.smithi062.sacpds on smithi062 2024-03-24T11:25:09.108 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:25:08 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:25:08.658Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"smithi062:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"172.21.15.139:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-24T11:25:10.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:10.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:10.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:09 smithi139 ceph-mon[91894]: Reconfiguring rgw.smpl.smithi062.xvpxnk (monmap changed)... 2024-03-24T11:25:10.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi062.xvpxnk", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-03-24T11:25:10.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:10.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:09 smithi139 ceph-mon[91894]: Reconfiguring daemon rgw.smpl.smithi062.xvpxnk on smithi062 2024-03-24T11:25:10.146 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:10.146 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:10.146 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:09 smithi062 ceph-mon[98643]: Reconfiguring rgw.smpl.smithi062.xvpxnk (monmap changed)... 2024-03-24T11:25:10.146 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi062.xvpxnk", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-03-24T11:25:10.146 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:10.147 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:09 smithi062 ceph-mon[98643]: Reconfiguring daemon rgw.smpl.smithi062.xvpxnk on smithi062 2024-03-24T11:25:10.147 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:10.147 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:10.147 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:09 smithi062 ceph-mon[90553]: Reconfiguring rgw.smpl.smithi062.xvpxnk (monmap changed)... 2024-03-24T11:25:10.147 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi062.xvpxnk", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-03-24T11:25:10.147 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:10.147 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:09 smithi062 ceph-mon[90553]: Reconfiguring daemon rgw.smpl.smithi062.xvpxnk on smithi062 2024-03-24T11:25:11.205 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:10 smithi139 ceph-mon[91894]: pgmap v47: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:25:11.205 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:11.205 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:11.205 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:11.206 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:11.206 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:11.206 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T11:25:11.206 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T11:25:11.206 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:11.226 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:25:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:25:10] "GET /metrics HTTP/1.1" 200 38269 "" "Prometheus/2.33.4" 2024-03-24T11:25:11.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:10 smithi062 ceph-mon[90553]: pgmap v47: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:25:11.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:11.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:11.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:11.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:11.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:11.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T11:25:11.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T11:25:11.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:11.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:10 smithi062 ceph-mon[98643]: pgmap v47: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:25:11.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:11.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:11.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:11.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:11.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:11.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T11:25:11.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T11:25:11.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:12.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:11 smithi139 ceph-mon[91894]: Reconfiguring mon.b (monmap changed)... 2024-03-24T11:25:12.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:11 smithi139 ceph-mon[91894]: Reconfiguring daemon mon.b on smithi139 2024-03-24T11:25:12.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:25:12.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:12.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:12.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:12.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:11 smithi062 ceph-mon[90553]: Reconfiguring mon.b (monmap changed)... 2024-03-24T11:25:12.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:11 smithi062 ceph-mon[90553]: Reconfiguring daemon mon.b on smithi139 2024-03-24T11:25:12.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:25:12.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:12.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:12.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:12.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:11 smithi062 ceph-mon[98643]: Reconfiguring mon.b (monmap changed)... 2024-03-24T11:25:12.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:11 smithi062 ceph-mon[98643]: Reconfiguring daemon mon.b on smithi139 2024-03-24T11:25:12.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:25:12.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:12.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:12.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:13.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:12 smithi139 ceph-mon[91894]: pgmap v48: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:25:13.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:13.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:13.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T11:25:13.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T11:25:13.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:13.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:12 smithi062 ceph-mon[98643]: pgmap v48: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:25:13.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:13.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:13.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T11:25:13.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T11:25:13.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:13.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:12 smithi062 ceph-mon[90553]: pgmap v48: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:25:13.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:13.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:13.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T11:25:13.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T11:25:13.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:14.201 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:13 smithi139 ceph-mon[91894]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:25:14.201 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:13 smithi139 ceph-mon[91894]: Reconfiguring mgr.x (monmap changed)... 2024-03-24T11:25:14.202 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:13 smithi139 ceph-mon[91894]: Reconfiguring daemon mgr.x on smithi139 2024-03-24T11:25:14.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:13 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:25:14.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:13 smithi062 ceph-mon[90553]: Reconfiguring mgr.x (monmap changed)... 2024-03-24T11:25:14.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:13 smithi062 ceph-mon[90553]: Reconfiguring daemon mgr.x on smithi139 2024-03-24T11:25:14.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:13 smithi062 ceph-mon[98643]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:25:14.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:13 smithi062 ceph-mon[98643]: Reconfiguring mgr.x (monmap changed)... 2024-03-24T11:25:14.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:13 smithi062 ceph-mon[98643]: Reconfiguring daemon mgr.x on smithi139 2024-03-24T11:25:15.174 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:14 smithi139 ceph-mon[91894]: pgmap v49: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:25:15.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:14 smithi062 ceph-mon[90553]: pgmap v49: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:25:15.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:14 smithi062 ceph-mon[98643]: pgmap v49: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:25:17.097 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:17 smithi139 ceph-mon[91894]: pgmap v50: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:25:17.097 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:17.097 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:17.097 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-24T11:25:17.098 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:17.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:17 smithi062 ceph-mon[90553]: pgmap v50: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:25:17.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:17.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:17.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-24T11:25:17.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:17.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:17 smithi062 ceph-mon[98643]: pgmap v50: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:25:17.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:17.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:17.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-24T11:25:17.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:18.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:18 smithi139 ceph-mon[91894]: Reconfiguring osd.4 (monmap changed)... 2024-03-24T11:25:18.074 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:18 smithi139 ceph-mon[91894]: Reconfiguring daemon osd.4 on smithi139 2024-03-24T11:25:18.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:18 smithi062 ceph-mon[90553]: Reconfiguring osd.4 (monmap changed)... 2024-03-24T11:25:18.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:18 smithi062 ceph-mon[90553]: Reconfiguring daemon osd.4 on smithi139 2024-03-24T11:25:18.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:18 smithi062 ceph-mon[98643]: Reconfiguring osd.4 (monmap changed)... 2024-03-24T11:25:18.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:18 smithi062 ceph-mon[98643]: Reconfiguring daemon osd.4 on smithi139 2024-03-24T11:25:18.979 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:25:18 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:25:18.659Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"smithi062:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"172.21.15.139:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-24T11:25:19.235 INFO:teuthology.orchestra.run.smithi062.stdout:true 2024-03-24T11:25:19.271 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:19 smithi139 ceph-mon[91894]: pgmap v51: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:25:19.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:19 smithi062 ceph-mon[90553]: pgmap v51: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:25:19.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:19 smithi062 ceph-mon[98643]: pgmap v51: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:25:19.906 INFO:teuthology.orchestra.run.smithi062.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T11:25:19.906 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi062 smithi062 running 9s ago 14m - - 2024-03-24T11:25:19.907 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi139 smithi139 running 8s ago 14m - - 2024-03-24T11:25:19.907 INFO:teuthology.orchestra.run.smithi062.stdout:alertmanager.a smithi062 *:9093,9094 running (4m) 9s ago 9m 25.1M - 0.23.0 ba2b418f427c 818298d58140 2024-03-24T11:25:19.907 INFO:teuthology.orchestra.run.smithi062.stdout:grafana.a smithi139 *:3000 running (3m) 8s ago 9m 53.3M - 8.3.5 dad864ee21e9 b48167e7171e 2024-03-24T11:25:19.907 INFO:teuthology.orchestra.run.smithi062.stdout:iscsi.foo.smithi062.dggsvk smithi062 running (2m) 9s ago 8m 56.6M - 3.5 e1d6a67b021e 0a98806ec839 2024-03-24T11:25:19.907 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.x smithi139 *:8443,9283,8765 running (2m) 8s ago 13m 448M - 19.0.0-1667-gdb0330b1 4f325f8440e9 d94987b11387 2024-03-24T11:25:19.907 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.y smithi062 *:8443,9283,8765 running (3m) 9s ago 15m 509M - 19.0.0-1667-gdb0330b1 4f325f8440e9 ca1a49028d65 2024-03-24T11:25:19.907 INFO:teuthology.orchestra.run.smithi062.stdout:mon.a smithi062 running (81s) 9s ago 15m 48.3M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 f7afe6e28c88 2024-03-24T11:25:19.907 INFO:teuthology.orchestra.run.smithi062.stdout:mon.b smithi139 running (14m) 8s ago 14m 61.2M 2048M 17.2.0 e1d6a67b021e 8945cdb4e6aa 2024-03-24T11:25:19.907 INFO:teuthology.orchestra.run.smithi062.stdout:mon.c smithi062 running (47s) 9s ago 14m 41.8M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 459e7449ec39 2024-03-24T11:25:19.907 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.a smithi062 *:9100 running (4m) 9s ago 10m 17.0M - 1.3.1 1dbe0e931976 79d5a02a999f 2024-03-24T11:25:19.907 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.b smithi139 *:9100 running (4m) 8s ago 10m 15.4M - 1.3.1 1dbe0e931976 8f9f4608e465 2024-03-24T11:25:19.907 INFO:teuthology.orchestra.run.smithi062.stdout:osd.0 smithi062 running (13m) 9s ago 13m 54.1M 4096M 17.2.0 e1d6a67b021e 8040b30d7170 2024-03-24T11:25:19.907 INFO:teuthology.orchestra.run.smithi062.stdout:osd.1 smithi062 running (13m) 9s ago 13m 55.6M 4096M 17.2.0 e1d6a67b021e e62d22e5dc29 2024-03-24T11:25:19.907 INFO:teuthology.orchestra.run.smithi062.stdout:osd.2 smithi062 running (12m) 9s ago 12m 49.2M 4096M 17.2.0 e1d6a67b021e 714f550103a7 2024-03-24T11:25:19.908 INFO:teuthology.orchestra.run.smithi062.stdout:osd.3 smithi062 running (12m) 9s ago 12m 53.8M 4096M 17.2.0 e1d6a67b021e 1d2868bd5195 2024-03-24T11:25:19.908 INFO:teuthology.orchestra.run.smithi062.stdout:osd.4 smithi139 running (12m) 8s ago 12m 57.9M 4096M 17.2.0 e1d6a67b021e 7568bfa9e74d 2024-03-24T11:25:19.908 INFO:teuthology.orchestra.run.smithi062.stdout:osd.5 smithi139 running (11m) 8s ago 11m 51.1M 4096M 17.2.0 e1d6a67b021e dd0abd599a14 2024-03-24T11:25:19.908 INFO:teuthology.orchestra.run.smithi062.stdout:osd.6 smithi139 running (11m) 8s ago 11m 50.7M 4096M 17.2.0 e1d6a67b021e a4b1289aca69 2024-03-24T11:25:19.908 INFO:teuthology.orchestra.run.smithi062.stdout:osd.7 smithi139 running (10m) 8s ago 10m 53.4M 4096M 17.2.0 e1d6a67b021e bc7fadbc1384 2024-03-24T11:25:19.908 INFO:teuthology.orchestra.run.smithi062.stdout:prometheus.a smithi139 *:9095 running (3m) 8s ago 10m 60.1M - 2.33.4 514e6a882f6e 2ed47f810d33 2024-03-24T11:25:19.908 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi062.sacpds smithi062 *:8000 running (8m) 9s ago 8m 73.2M - 17.2.0 e1d6a67b021e 89f4ac308e98 2024-03-24T11:25:19.908 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi139.yljesh smithi139 *:8000 running (8m) 8s ago 8m 71.8M - 17.2.0 e1d6a67b021e e189bdc12cae 2024-03-24T11:25:19.908 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi062.xvpxnk smithi062 *:80 running (8m) 9s ago 8m 73.0M - 17.2.0 e1d6a67b021e d1aa74adab44 2024-03-24T11:25:19.908 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi139.pbtrmp smithi139 *:80 running (8m) 8s ago 8m 75.2M - 17.2.0 e1d6a67b021e 7f8e336755d7 2024-03-24T11:25:20.314 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:25:20.314 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": { 2024-03-24T11:25:20.314 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 3 2024-03-24T11:25:20.314 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:25:20.314 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": { 2024-03-24T11:25:20.315 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-24T11:25:20.315 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:25:20.315 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": { 2024-03-24T11:25:20.315 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 8 2024-03-24T11:25:20.315 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:25:20.315 INFO:teuthology.orchestra.run.smithi062.stdout: "rgw": { 2024-03-24T11:25:20.315 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4 2024-03-24T11:25:20.315 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:25:20.315 INFO:teuthology.orchestra.run.smithi062.stdout: "overall": { 2024-03-24T11:25:20.315 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 12, 2024-03-24T11:25:20.315 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 5 2024-03-24T11:25:20.315 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-24T11:25:20.315 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:25:20.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:20 smithi139 ceph-mon[91894]: pgmap v52: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:25:20.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:20 smithi139 ceph-mon[91894]: from='client.64138 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:25:20.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:20.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:20.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:20 smithi139 ceph-mon[91894]: Reconfiguring osd.5 (monmap changed)... 2024-03-24T11:25:20.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-24T11:25:20.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:20.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:20 smithi139 ceph-mon[91894]: Reconfiguring daemon osd.5 on smithi139 2024-03-24T11:25:20.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:20 smithi139 ceph-mon[91894]: from='client.64144 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:25:20.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:20 smithi139 ceph-mon[91894]: from='client.44184 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:25:20.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:20 smithi139 ceph-mon[91894]: from='client.? 172.21.15.62:0/471953484' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:20.650 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:25:20.651 INFO:teuthology.orchestra.run.smithi062.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69", 2024-03-24T11:25:20.651 INFO:teuthology.orchestra.run.smithi062.stdout: "in_progress": true, 2024-03-24T11:25:20.651 INFO:teuthology.orchestra.run.smithi062.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T11:25:20.651 INFO:teuthology.orchestra.run.smithi062.stdout: "services_complete": [ 2024-03-24T11:25:20.651 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr" 2024-03-24T11:25:20.651 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-03-24T11:25:20.651 INFO:teuthology.orchestra.run.smithi062.stdout: "progress": "4/23 daemons upgraded", 2024-03-24T11:25:20.651 INFO:teuthology.orchestra.run.smithi062.stdout: "message": "Currently upgrading mon daemons", 2024-03-24T11:25:20.651 INFO:teuthology.orchestra.run.smithi062.stdout: "is_paused": false 2024-03-24T11:25:20.651 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:25:20.652 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:20 smithi062 ceph-mon[98643]: pgmap v52: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:25:20.652 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:20 smithi062 ceph-mon[98643]: from='client.64138 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:25:20.652 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:20.652 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:20.652 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:20 smithi062 ceph-mon[98643]: Reconfiguring osd.5 (monmap changed)... 2024-03-24T11:25:20.652 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-24T11:25:20.652 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:20.652 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:20 smithi062 ceph-mon[98643]: Reconfiguring daemon osd.5 on smithi139 2024-03-24T11:25:20.652 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:20 smithi062 ceph-mon[98643]: from='client.64144 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:25:20.652 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:20 smithi062 ceph-mon[98643]: from='client.44184 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:25:20.652 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:20 smithi062 ceph-mon[98643]: from='client.? 172.21.15.62:0/471953484' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:20.653 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:20 smithi062 ceph-mon[90553]: pgmap v52: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:25:20.653 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:20 smithi062 ceph-mon[90553]: from='client.64138 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:25:20.653 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:20.653 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:20.653 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:20 smithi062 ceph-mon[90553]: Reconfiguring osd.5 (monmap changed)... 2024-03-24T11:25:20.653 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-24T11:25:20.653 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:20.653 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:20 smithi062 ceph-mon[90553]: Reconfiguring daemon osd.5 on smithi139 2024-03-24T11:25:20.653 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:20 smithi062 ceph-mon[90553]: from='client.64144 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:25:20.653 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:20 smithi062 ceph-mon[90553]: from='client.44184 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:25:20.653 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:20 smithi062 ceph-mon[90553]: from='client.? 172.21.15.62:0/471953484' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:20.976 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:25:20 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:25:20] "GET /metrics HTTP/1.1" 200 38266 "" "Prometheus/2.33.4" 2024-03-24T11:25:21.057 INFO:teuthology.orchestra.run.smithi062.stdout:HEALTH_OK 2024-03-24T11:25:21.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:21 smithi062 ceph-mon[90553]: from='client.64162 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:25:21.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:21 smithi062 ceph-mon[90553]: from='client.? 172.21.15.62:0/2131560688' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:25:21.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:21 smithi062 ceph-mon[98643]: from='client.64162 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:25:21.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:21 smithi062 ceph-mon[98643]: from='client.? 172.21.15.62:0/2131560688' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:25:21.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:21 smithi139 ceph-mon[91894]: from='client.64162 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:25:21.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:21 smithi139 ceph-mon[91894]: from='client.? 172.21.15.62:0/2131560688' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:25:22.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:22 smithi139 ceph-mon[91894]: pgmap v53: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:25:22.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:22.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:22.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:22 smithi139 ceph-mon[91894]: Reconfiguring osd.6 (monmap changed)... 2024-03-24T11:25:22.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-24T11:25:22.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:22.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:22 smithi139 ceph-mon[91894]: Reconfiguring daemon osd.6 on smithi139 2024-03-24T11:25:22.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:22.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:22.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:22.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:22 smithi062 ceph-mon[98643]: pgmap v53: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:25:22.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:22.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:22.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:22 smithi062 ceph-mon[98643]: Reconfiguring osd.6 (monmap changed)... 2024-03-24T11:25:22.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-24T11:25:22.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:22.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:22 smithi062 ceph-mon[98643]: Reconfiguring daemon osd.6 on smithi139 2024-03-24T11:25:22.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:22.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:22.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:22.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:22 smithi062 ceph-mon[90553]: pgmap v53: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:25:22.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:22.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:22.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:22 smithi062 ceph-mon[90553]: Reconfiguring osd.6 (monmap changed)... 2024-03-24T11:25:22.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-24T11:25:22.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:22.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:22 smithi062 ceph-mon[90553]: Reconfiguring daemon osd.6 on smithi139 2024-03-24T11:25:22.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:22.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:22.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:23.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:23 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:25:23.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:23.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:23.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-24T11:25:23.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:23.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:23 smithi062 ceph-mon[98643]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:25:23.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:23.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:23.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-24T11:25:23.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:23.781 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:23 smithi139 ceph-mon[91894]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:25:23.782 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:23.782 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:23.782 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-24T11:25:23.782 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:24.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:24 smithi062 ceph-mon[98643]: pgmap v54: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:25:24.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:24 smithi062 ceph-mon[98643]: Reconfiguring osd.7 (monmap changed)... 2024-03-24T11:25:24.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:24 smithi062 ceph-mon[98643]: Reconfiguring daemon osd.7 on smithi139 2024-03-24T11:25:24.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:24 smithi062 ceph-mon[90553]: pgmap v54: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:25:24.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:24 smithi062 ceph-mon[90553]: Reconfiguring osd.7 (monmap changed)... 2024-03-24T11:25:24.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:24 smithi062 ceph-mon[90553]: Reconfiguring daemon osd.7 on smithi139 2024-03-24T11:25:24.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:24 smithi139 ceph-mon[91894]: pgmap v54: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:25:24.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:24 smithi139 ceph-mon[91894]: Reconfiguring osd.7 (monmap changed)... 2024-03-24T11:25:24.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:24 smithi139 ceph-mon[91894]: Reconfiguring daemon osd.7 on smithi139 2024-03-24T11:25:25.838 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:25.838 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:25.838 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:25 smithi139 ceph-mon[91894]: Reconfiguring rgw.foo.smithi139.yljesh (monmap changed)... 2024-03-24T11:25:25.839 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi139.yljesh", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-03-24T11:25:25.839 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:25.839 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:25 smithi139 ceph-mon[91894]: Reconfiguring daemon rgw.foo.smithi139.yljesh on smithi139 2024-03-24T11:25:25.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:25.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:25.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:25 smithi062 ceph-mon[98643]: Reconfiguring rgw.foo.smithi139.yljesh (monmap changed)... 2024-03-24T11:25:25.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi139.yljesh", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-03-24T11:25:25.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:25.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:25 smithi062 ceph-mon[98643]: Reconfiguring daemon rgw.foo.smithi139.yljesh on smithi139 2024-03-24T11:25:25.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:25.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:25.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:25 smithi062 ceph-mon[90553]: Reconfiguring rgw.foo.smithi139.yljesh (monmap changed)... 2024-03-24T11:25:25.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi139.yljesh", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-03-24T11:25:25.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:25.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:25 smithi062 ceph-mon[90553]: Reconfiguring daemon rgw.foo.smithi139.yljesh on smithi139 2024-03-24T11:25:26.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:26 smithi139 ceph-mon[91894]: pgmap v55: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:25:26.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:25:26.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:26.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:26.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi139.pbtrmp", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-03-24T11:25:26.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:26.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:26 smithi062 ceph-mon[90553]: pgmap v55: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:25:26.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:25:26.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:26.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:26.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi139.pbtrmp", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-03-24T11:25:26.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:26.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:26 smithi062 ceph-mon[98643]: pgmap v55: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:25:26.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:25:26.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:26.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:26.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi139.pbtrmp", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-03-24T11:25:26.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:27.976 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:27 smithi062 ceph-mon[90553]: Reconfiguring rgw.smpl.smithi139.pbtrmp (monmap changed)... 2024-03-24T11:25:27.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:27 smithi062 ceph-mon[90553]: Reconfiguring daemon rgw.smpl.smithi139.pbtrmp on smithi139 2024-03-24T11:25:27.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:27 smithi062 ceph-mon[98643]: Reconfiguring rgw.smpl.smithi139.pbtrmp (monmap changed)... 2024-03-24T11:25:27.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:27 smithi062 ceph-mon[98643]: Reconfiguring daemon rgw.smpl.smithi139.pbtrmp on smithi139 2024-03-24T11:25:28.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:27 smithi139 ceph-mon[91894]: Reconfiguring rgw.smpl.smithi139.pbtrmp (monmap changed)... 2024-03-24T11:25:28.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:27 smithi139 ceph-mon[91894]: Reconfiguring daemon rgw.smpl.smithi139.pbtrmp on smithi139 2024-03-24T11:25:28.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:28 smithi062 ceph-mon[90553]: pgmap v56: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:25:28.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:28.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:28.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:28.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:28.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:28.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:28 smithi062 ceph-mon[90553]: Upgrade: Setting container_image for all mon 2024-03-24T11:25:28.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:28.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2024-03-24T11:25:28.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]': finished 2024-03-24T11:25:28.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]: dispatch 2024-03-24T11:25:28.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]': finished 2024-03-24T11:25:28.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2024-03-24T11:25:28.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]': finished 2024-03-24T11:25:28.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:28.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:28 smithi062 ceph-mon[90553]: Upgrade: Setting container_image for all crash 2024-03-24T11:25:28.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:28.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-24T11:25:28.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:28 smithi062 ceph-mon[90553]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-24T11:25:28.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:28 smithi062 ceph-mon[90553]: Upgrade: osd.0 is safe to restart 2024-03-24T11:25:28.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:28 smithi062 ceph-mon[98643]: pgmap v56: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:25:28.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:28.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:28.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:28.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:28.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:28.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:28 smithi062 ceph-mon[98643]: Upgrade: Setting container_image for all mon 2024-03-24T11:25:28.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:28.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2024-03-24T11:25:28.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]': finished 2024-03-24T11:25:28.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]: dispatch 2024-03-24T11:25:28.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]': finished 2024-03-24T11:25:28.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2024-03-24T11:25:28.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]': finished 2024-03-24T11:25:28.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:28.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:28 smithi062 ceph-mon[98643]: Upgrade: Setting container_image for all crash 2024-03-24T11:25:28.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:28.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-24T11:25:28.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:28 smithi062 ceph-mon[98643]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-24T11:25:28.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:28 smithi062 ceph-mon[98643]: Upgrade: osd.0 is safe to restart 2024-03-24T11:25:29.108 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:25:28 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:25:28.660Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"smithi062:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"172.21.15.139:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-24T11:25:29.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:28 smithi139 ceph-mon[91894]: pgmap v56: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:25:29.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:29.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:29.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:29.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:29.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:29.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:28 smithi139 ceph-mon[91894]: Upgrade: Setting container_image for all mon 2024-03-24T11:25:29.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:29.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2024-03-24T11:25:29.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]': finished 2024-03-24T11:25:29.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]: dispatch 2024-03-24T11:25:29.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]': finished 2024-03-24T11:25:29.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2024-03-24T11:25:29.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]': finished 2024-03-24T11:25:29.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:29.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:28 smithi139 ceph-mon[91894]: Upgrade: Setting container_image for all crash 2024-03-24T11:25:29.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:29.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-24T11:25:29.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:28 smithi139 ceph-mon[91894]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-24T11:25:29.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:28 smithi139 ceph-mon[91894]: Upgrade: osd.0 is safe to restart 2024-03-24T11:25:30.687 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:30 smithi062 ceph-mon[98643]: pgmap v57: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:25:30.687 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:30 smithi062 ceph-mon[98643]: Upgrade: Updating osd.0 2024-03-24T11:25:30.687 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:30.687 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-24T11:25:30.687 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:30.687 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:30 smithi062 ceph-mon[98643]: Deploying daemon osd.0 on smithi062 2024-03-24T11:25:30.687 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:30.687 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:30.688 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:30.688 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:30 smithi062 ceph-mon[90553]: pgmap v57: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:25:30.688 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:30 smithi062 ceph-mon[90553]: Upgrade: Updating osd.0 2024-03-24T11:25:30.688 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:30.688 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-24T11:25:30.688 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:30.688 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:30 smithi062 ceph-mon[90553]: Deploying daemon osd.0 on smithi062 2024-03-24T11:25:30.688 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:30.688 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:30.688 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:30.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:30 smithi139 ceph-mon[91894]: pgmap v57: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:25:30.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:30 smithi139 ceph-mon[91894]: Upgrade: Updating osd.0 2024-03-24T11:25:30.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:30.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-24T11:25:30.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:30.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:30 smithi139 ceph-mon[91894]: Deploying daemon osd.0 on smithi062 2024-03-24T11:25:30.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:30.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:30.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:30.977 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:25:30 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:25:30] "GET /metrics HTTP/1.1" 200 38338 "" "Prometheus/2.33.4" 2024-03-24T11:25:32.723 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:32 smithi062 ceph-mon[90553]: pgmap v58: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:25:32.724 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:32 smithi062 ceph-mon[98643]: pgmap v58: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:25:32.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:32 smithi139 ceph-mon[91894]: pgmap v58: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:25:32.976 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:32 smithi062 systemd[1]: Stopping Ceph osd.0 for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:25:33.726 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:33 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:25:33.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:33 smithi062 ceph-mon[98643]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:25:33.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:33 smithi139 ceph-mon[91894]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:25:34.227 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0[32974]: 2024-03-24T11:25:33.988+0000 7fa9dabdc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T11:25:34.227 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0[32974]: 2024-03-24T11:25:33.988+0000 7fa9dabdc700 -1 osd.0 108 *** Got signal Terminated *** 2024-03-24T11:25:34.227 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0[32974]: 2024-03-24T11:25:33.988+0000 7fa9dabdc700 -1 osd.0 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T11:25:34.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:34 smithi139 ceph-mon[91894]: pgmap v59: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:25:34.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:34 smithi139 ceph-mon[91894]: osd.0 marked itself down and dead 2024-03-24T11:25:34.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:34 smithi062 ceph-mon[98643]: pgmap v59: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:25:34.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:34 smithi062 ceph-mon[98643]: osd.0 marked itself down and dead 2024-03-24T11:25:34.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:34 smithi062 ceph-mon[90553]: pgmap v59: 161 pgs: 161 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:25:34.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:34 smithi062 ceph-mon[90553]: osd.0 marked itself down and dead 2024-03-24T11:25:35.227 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:34 smithi062 podman[107182]: 2024-03-24 11:25:34.941757788 +0000 UTC m=+1.053427122 container died 8040b30d7170d5c8f1555bebad97929d4701d0963161ef960c78a9f4c0dd67a0 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0, ceph=True, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, io.openshift.tags=base centos centos-stream, vendor=Red Hat, Inc., CEPH_POINT_RELEASE=-17.2.0, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, com.redhat.component=centos-stream-container, io.buildah.version=1.19.8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.openshift.expose-services=, GIT_REPO=https://github.com/ceph/ceph-container.git, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, distribution-scope=public, build-date=2022-05-03T08:36:31.336870, io.k8s.display-name=CentOS Stream 8, GIT_BRANCH=HEAD, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, vcs-type=git, RELEASE=HEAD, architecture=x86_64, name=centos-stream, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, version=8, release=754) 2024-03-24T11:25:35.616 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:35 smithi062 ceph-mon[98643]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T11:25:35.616 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:35 smithi062 ceph-mon[98643]: failed to encode map e109 with expected crc 2024-03-24T11:25:35.617 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:35 smithi062 ceph-mon[98643]: failed to encode map e109 with expected crc 2024-03-24T11:25:35.617 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:35 smithi062 ceph-mon[98643]: osdmap e109: 8 total, 7 up, 8 in 2024-03-24T11:25:35.617 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:35 smithi062 ceph-mon[98643]: failed to encode map e109 with expected crc 2024-03-24T11:25:35.617 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:35 smithi062 ceph-mon[98643]: failed to encode map e109 with expected crc 2024-03-24T11:25:35.617 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:35 smithi062 ceph-mon[98643]: failed to encode map e109 with expected crc 2024-03-24T11:25:35.617 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:35 smithi062 ceph-mon[90553]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T11:25:35.617 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:35 smithi062 ceph-mon[90553]: failed to encode map e109 with expected crc 2024-03-24T11:25:35.617 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:35 smithi062 ceph-mon[90553]: failed to encode map e109 with expected crc 2024-03-24T11:25:35.617 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:35 smithi062 ceph-mon[90553]: osdmap e109: 8 total, 7 up, 8 in 2024-03-24T11:25:35.617 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:35 smithi062 ceph-mon[90553]: failed to encode map e109 with expected crc 2024-03-24T11:25:35.618 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:35 smithi062 ceph-mon[90553]: failed to encode map e109 with expected crc 2024-03-24T11:25:35.618 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:35 smithi062 ceph-mon[90553]: failed to encode map e109 with expected crc 2024-03-24T11:25:35.618 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:35 smithi062 podman[107182]: 2024-03-24 11:25:35.381642704 +0000 UTC m=+1.493312065 container cleanup 8040b30d7170d5c8f1555bebad97929d4701d0963161ef960c78a9f4c0dd67a0 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0, release=754, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, distribution-scope=public, GIT_CLEAN=True, io.openshift.expose-services=, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., architecture=x86_64, io.buildah.version=1.19.8, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, ceph=True, vendor=Red Hat, Inc., version=8, build-date=2022-05-03T08:36:31.336870, CEPH_POINT_RELEASE=-17.2.0, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, io.openshift.tags=base centos centos-stream, name=centos-stream, vcs-type=git, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, com.redhat.component=centos-stream-container, RELEASE=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, io.k8s.display-name=CentOS Stream 8) 2024-03-24T11:25:35.618 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:35 smithi062 bash[107182]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0 2024-03-24T11:25:35.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:35 smithi139 ceph-mon[91894]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T11:25:35.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:35 smithi139 ceph-mon[91894]: failed to encode map e109 with expected crc 2024-03-24T11:25:35.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:35 smithi139 ceph-mon[91894]: failed to encode map e109 with expected crc 2024-03-24T11:25:35.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:35 smithi139 ceph-mon[91894]: osdmap e109: 8 total, 7 up, 8 in 2024-03-24T11:25:35.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:35 smithi139 ceph-mon[91894]: failed to encode map e109 with expected crc 2024-03-24T11:25:35.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:35 smithi139 ceph-mon[91894]: failed to encode map e109 with expected crc 2024-03-24T11:25:35.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:35 smithi139 ceph-mon[91894]: failed to encode map e109 with expected crc 2024-03-24T11:25:35.977 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:35 smithi062 podman[107199]: 2024-03-24 11:25:35.616776123 +0000 UTC m=+0.670296736 container remove 8040b30d7170d5c8f1555bebad97929d4701d0963161ef960c78a9f4c0dd67a0 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, distribution-scope=public, RELEASE=HEAD, architecture=x86_64, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, io.openshift.tags=base centos centos-stream, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-17.2.0, build-date=2022-05-03T08:36:31.336870, io.k8s.display-name=CentOS Stream 8, GIT_CLEAN=True, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, vcs-type=git, io.buildah.version=1.19.8, release=754, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, name=centos-stream, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, GIT_BRANCH=HEAD, io.openshift.expose-services=, ceph=True, version=8, com.redhat.component=centos-stream-container, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vendor=Red Hat, Inc., description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream) 2024-03-24T11:25:36.320 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:36 smithi062 podman[107290]: 2024-03-24 11:25:36.043204812 +0000 UTC m=+0.109887557 container create 3cb41f2d918089115648ce93182126f1acb31e6dda90b4ba5236bd9e69225bff (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-db0330b, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T11:25:36.321 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:36 smithi062 podman[107290]: 2024-03-24 11:25:35.95131127 +0000 UTC m=+0.017993977 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:25:36.321 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:36 smithi062 podman[107290]: 2024-03-24 11:25:36.262190907 +0000 UTC m=+0.328873608 container init 3cb41f2d918089115648ce93182126f1acb31e6dda90b4ba5236bd9e69225bff (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-24T11:25:36.321 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:36 smithi062 podman[107290]: 2024-03-24 11:25:36.265416212 +0000 UTC m=+0.332098916 container start 3cb41f2d918089115648ce93182126f1acb31e6dda90b4ba5236bd9e69225bff (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-24T11:25:36.321 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:36 smithi062 podman[107290]: 2024-03-24 11:25:36.320697333 +0000 UTC m=+0.387380045 container attach 3cb41f2d918089115648ce93182126f1acb31e6dda90b4ba5236bd9e69225bff (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-db0330b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-24T11:25:36.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:36 smithi139 ceph-mon[91894]: pgmap v61: 161 pgs: 21 stale+active+clean, 140 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T11:25:36.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:36 smithi139 ceph-mon[91894]: failed to encode map e110 with expected crc 2024-03-24T11:25:36.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:36 smithi139 ceph-mon[91894]: failed to encode map e110 with expected crc 2024-03-24T11:25:36.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:36 smithi139 ceph-mon[91894]: osdmap e110: 8 total, 7 up, 8 in 2024-03-24T11:25:36.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:36 smithi139 ceph-mon[91894]: failed to encode map e110 with expected crc 2024-03-24T11:25:36.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:36 smithi139 ceph-mon[91894]: failed to encode map e110 with expected crc 2024-03-24T11:25:36.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:36 smithi139 ceph-mon[91894]: failed to encode map e110 with expected crc 2024-03-24T11:25:36.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:36 smithi139 ceph-mon[91894]: failed to encode map e110 with expected crc 2024-03-24T11:25:36.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:36 smithi139 ceph-mon[91894]: failed to encode map e110 with expected crc 2024-03-24T11:25:36.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:36 smithi139 ceph-mon[91894]: failed to encode map e110 with expected crc 2024-03-24T11:25:36.727 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:36 smithi062 podman[107290]: 2024-03-24 11:25:36.56392923 +0000 UTC m=+0.630611931 container died 3cb41f2d918089115648ce93182126f1acb31e6dda90b4ba5236bd9e69225bff (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, RELEASE=squid-db0330b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T11:25:36.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:36 smithi062 ceph-mon[90553]: pgmap v61: 161 pgs: 21 stale+active+clean, 140 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T11:25:36.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:36 smithi062 ceph-mon[90553]: failed to encode map e110 with expected crc 2024-03-24T11:25:36.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:36 smithi062 ceph-mon[90553]: failed to encode map e110 with expected crc 2024-03-24T11:25:36.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:36 smithi062 ceph-mon[90553]: osdmap e110: 8 total, 7 up, 8 in 2024-03-24T11:25:36.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:36 smithi062 ceph-mon[90553]: failed to encode map e110 with expected crc 2024-03-24T11:25:36.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:36 smithi062 ceph-mon[90553]: failed to encode map e110 with expected crc 2024-03-24T11:25:36.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:36 smithi062 ceph-mon[90553]: failed to encode map e110 with expected crc 2024-03-24T11:25:36.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:36 smithi062 ceph-mon[90553]: failed to encode map e110 with expected crc 2024-03-24T11:25:36.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:36 smithi062 ceph-mon[90553]: failed to encode map e110 with expected crc 2024-03-24T11:25:36.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:36 smithi062 ceph-mon[90553]: failed to encode map e110 with expected crc 2024-03-24T11:25:36.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:36 smithi062 ceph-mon[98643]: pgmap v61: 161 pgs: 21 stale+active+clean, 140 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T11:25:36.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:36 smithi062 ceph-mon[98643]: failed to encode map e110 with expected crc 2024-03-24T11:25:36.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:36 smithi062 ceph-mon[98643]: failed to encode map e110 with expected crc 2024-03-24T11:25:36.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:36 smithi062 ceph-mon[98643]: osdmap e110: 8 total, 7 up, 8 in 2024-03-24T11:25:36.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:36 smithi062 ceph-mon[98643]: failed to encode map e110 with expected crc 2024-03-24T11:25:36.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:36 smithi062 ceph-mon[98643]: failed to encode map e110 with expected crc 2024-03-24T11:25:36.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:36 smithi062 ceph-mon[98643]: failed to encode map e110 with expected crc 2024-03-24T11:25:36.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:36 smithi062 ceph-mon[98643]: failed to encode map e110 with expected crc 2024-03-24T11:25:36.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:36 smithi062 ceph-mon[98643]: failed to encode map e110 with expected crc 2024-03-24T11:25:36.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:36 smithi062 ceph-mon[98643]: failed to encode map e110 with expected crc 2024-03-24T11:25:37.477 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:37 smithi062 podman[107315]: 2024-03-24 11:25:37.133200582 +0000 UTC m=+0.564570587 container remove 3cb41f2d918089115648ce93182126f1acb31e6dda90b4ba5236bd9e69225bff (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-db0330b, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-24T11:25:37.477 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:37 smithi062 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.0.service: Deactivated successfully. 2024-03-24T11:25:37.477 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:37 smithi062 systemd[1]: Stopped Ceph osd.0 for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:25:37.477 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:37 smithi062 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.0.service: Consumed 7.854s CPU time. 2024-03-24T11:25:37.477 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:37 smithi062 systemd[1]: Starting Ceph osd.0 for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:25:37.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:37 smithi139 ceph-mon[91894]: Health check failed: Degraded data redundancy: 68/630 objects degraded (10.794%), 20 pgs degraded (PG_DEGRADED) 2024-03-24T11:25:37.870 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:37 smithi062 ceph-mon[90553]: Health check failed: Degraded data redundancy: 68/630 objects degraded (10.794%), 20 pgs degraded (PG_DEGRADED) 2024-03-24T11:25:37.871 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:37 smithi062 ceph-mon[98643]: Health check failed: Degraded data redundancy: 68/630 objects degraded (10.794%), 20 pgs degraded (PG_DEGRADED) 2024-03-24T11:25:37.871 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:37 smithi062 podman[107422]: 2024-03-24 11:25:37.480725096 +0000 UTC m=+0.018316529 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:25:37.871 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:37 smithi062 podman[107422]: 2024-03-24 11:25:37.609881012 +0000 UTC m=+0.147472442 container create 91e0fd47f903f0a5dc45fb4c7137cdf719d08788b013894b723e14d6ceaae4c3 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0-activate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-db0330b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-24T11:25:38.227 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:37 smithi062 podman[107422]: 2024-03-24 11:25:37.87109574 +0000 UTC m=+0.408687183 container init 91e0fd47f903f0a5dc45fb4c7137cdf719d08788b013894b723e14d6ceaae4c3 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0-activate, RELEASE=squid-db0330b, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-03-24T11:25:38.227 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:37 smithi062 podman[107422]: 2024-03-24 11:25:37.8742879 +0000 UTC m=+0.411879340 container start 91e0fd47f903f0a5dc45fb4c7137cdf719d08788b013894b723e14d6ceaae4c3 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0-activate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-db0330b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T11:25:38.227 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:37 smithi062 podman[107422]: 2024-03-24 11:25:37.954418235 +0000 UTC m=+0.492009679 container attach 91e0fd47f903f0a5dc45fb4c7137cdf719d08788b013894b723e14d6ceaae4c3 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0-activate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-db0330b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True) 2024-03-24T11:25:38.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:38 smithi062 ceph-mon[90553]: pgmap v63: 161 pgs: 20 active+undersized+degraded, 38 active+undersized, 103 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s; 68/630 objects degraded (10.794%) 2024-03-24T11:25:38.625 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:38 smithi062 ceph-mon[98643]: pgmap v63: 161 pgs: 20 active+undersized+degraded, 38 active+undersized, 103 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s; 68/630 objects degraded (10.794%) 2024-03-24T11:25:38.626 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:38 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0-activate[107437]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-24T11:25:38.626 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:38 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0-activate[107437]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-24T11:25:38.626 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:38 smithi062 bash[107422]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-24T11:25:38.626 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:38 smithi062 bash[107422]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-24T11:25:38.626 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:38 smithi062 bash[107422]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-24T11:25:38.626 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:38 smithi062 bash[107422]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-24T11:25:38.626 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:38 smithi062 bash[107422]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-03-24T11:25:38.626 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:38 smithi062 bash[107422]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-24T11:25:38.626 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:38 smithi062 bash[107422]: --> ceph-volume raw activate successful for osd ID: 0 2024-03-24T11:25:38.626 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:38 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0-activate[107437]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-24T11:25:38.626 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:38 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0-activate[107437]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-24T11:25:38.626 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:38 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0-activate[107437]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-03-24T11:25:38.627 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:38 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0-activate[107437]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-24T11:25:38.627 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:38 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0-activate[107437]: --> ceph-volume raw activate successful for osd ID: 0 2024-03-24T11:25:38.627 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:38 smithi062 podman[107422]: 2024-03-24 11:25:38.344789254 +0000 UTC m=+0.882380677 container died 91e0fd47f903f0a5dc45fb4c7137cdf719d08788b013894b723e14d6ceaae4c3 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0-activate, ceph=True, org.label-schema.build-date=20240307, RELEASE=squid-db0330b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-24T11:25:38.781 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:38 smithi139 ceph-mon[91894]: pgmap v63: 161 pgs: 20 active+undersized+degraded, 38 active+undersized, 103 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s; 68/630 objects degraded (10.794%) 2024-03-24T11:25:38.782 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:25:38 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:25:38.659Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"smithi062:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"172.21.15.139:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-24T11:25:39.227 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:38 smithi062 podman[107574]: 2024-03-24 11:25:38.950670508 +0000 UTC m=+0.601040955 container remove 91e0fd47f903f0a5dc45fb4c7137cdf719d08788b013894b723e14d6ceaae4c3 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0-activate, GIT_CLEAN=True, RELEASE=squid-db0330b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T11:25:39.227 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:39 smithi062 podman[107640]: 2024-03-24 11:25:39.065043132 +0000 UTC m=+0.017890265 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:25:39.227 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:39 smithi062 podman[107640]: 2024-03-24 11:25:39.176809375 +0000 UTC m=+0.129656526 container create 61e78e75257c59caf926fe47afc07a9a349c3e5d30dfbeba6180bef63b5c0307 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-24T11:25:39.679 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:39 smithi062 podman[107640]: 2024-03-24 11:25:39.387352711 +0000 UTC m=+0.340199847 container init 61e78e75257c59caf926fe47afc07a9a349c3e5d30dfbeba6180bef63b5c0307 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b) 2024-03-24T11:25:39.679 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:39 smithi062 podman[107640]: 2024-03-24 11:25:39.39034925 +0000 UTC m=+0.343196350 container start 61e78e75257c59caf926fe47afc07a9a349c3e5d30dfbeba6180bef63b5c0307 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=) 2024-03-24T11:25:39.679 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:39 smithi062 ceph-osd[107659]: -- 172.21.15.62:0/3193991018 <== mon.1 v2:172.21.15.62:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562131abf1e0 con 0x562131a91400 2024-03-24T11:25:39.679 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:39 smithi062 bash[107640]: 61e78e75257c59caf926fe47afc07a9a349c3e5d30dfbeba6180bef63b5c0307 2024-03-24T11:25:39.679 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:39 smithi062 systemd[1]: Started Ceph osd.0 for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:25:39.936 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:39.936 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:39.936 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:39.936 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:39.936 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:39.936 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:39.936 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:39.936 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:39.937 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:39.937 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:39.937 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:39.937 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:39.937 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:39.937 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:39.937 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:39.937 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:39.938 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:39.938 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:39.938 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:39.938 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:39.938 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:39.938 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:39.938 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:39.938 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:39.938 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:39.938 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:40.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:40.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:40.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:40.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:40.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:40.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:40.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:40.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:40.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:40.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:40.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:40.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:40.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:40.226 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:39 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0[107655]: 2024-03-24T11:25:39.935+0000 7efc569e37c0 -1 Falling back to public interface 2024-03-24T11:25:40.977 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:25:40] "GET /metrics HTTP/1.1" 200 38338 "" "Prometheus/2.33.4" 2024-03-24T11:25:40.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[90553]: pgmap v64: 161 pgs: 20 active+undersized+degraded, 38 active+undersized, 103 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s; 68/630 objects degraded (10.794%) 2024-03-24T11:25:40.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:40.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:40.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:40.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:40.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:40.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:40.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:40.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:40.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:40.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:40.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:40.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:40.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:40.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:40.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:40.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:40.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:40.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:40.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:40.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:40.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:40.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:40.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:40.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:40.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:40.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:40.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:40.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:40.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:40.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:40.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:40.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:40.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:40.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:40.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:40.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:40.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:40.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:40.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:40.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:40.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:40.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:40.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:40.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:40.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[98643]: pgmap v64: 161 pgs: 20 active+undersized+degraded, 38 active+undersized, 103 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s; 68/630 objects degraded (10.794%) 2024-03-24T11:25:40.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:40.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:40.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:40.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:40.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:40.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:40.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:40.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:40.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:40.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:40.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:40.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:40.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:40.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:40.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:40.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:40.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:40.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:40.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:40.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:40.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:40.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:40.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:40.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:40.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:40.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:40.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:40.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:40.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:40.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:40.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:40.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:40.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:40.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:40.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:40.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:40.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:40.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:40.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:40.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:40.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:40.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:40.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:40.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:41.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:40 smithi139 ceph-mon[91894]: pgmap v64: 161 pgs: 20 active+undersized+degraded, 38 active+undersized, 103 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s; 68/630 objects degraded (10.794%) 2024-03-24T11:25:41.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:40 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:41.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:40 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:41.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:41.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:41.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:41.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:41.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:41.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:40 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:41.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:41.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:41.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:41.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:41.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:41.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:40 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:41.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:41.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:41.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:41.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:41.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:41.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:41.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:41.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:41.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:41.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:41.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:41.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:41.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:41.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:41.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:41.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:41.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:41.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:41.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:41.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:41.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:41.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:41.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:41.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:41.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:41.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:41.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:41.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:41.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:41.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:42.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:41 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:42.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:41 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:42.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:41 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:42.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:41 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:42.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:41 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:42.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:41 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:42.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:42.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:42.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:42.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:42.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:42.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:41 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:42.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:42.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:42.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:42.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:42.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:42.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:42.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:25:42.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:42.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:42.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:42.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:42.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:42.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:42.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:42.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:42.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:42.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:42.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:42.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:42.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:42.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:42.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:42.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:42.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:42.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:42.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:42.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:42.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:42.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:42.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:42.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:42.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:42.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:42.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:42.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:42.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:42.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:42.157 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:42.157 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:42.157 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:42.157 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:42.157 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:42.157 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:42.157 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:42.158 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:42.158 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:42.158 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:42.158 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:42.158 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:42.158 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:42.158 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:42.158 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:42.158 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:42.158 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:42.158 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:42.158 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:25:42.159 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:42.159 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:42.159 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:42.159 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:42.159 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:42.159 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:42.159 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:42.159 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:42.159 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:42.159 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:42.159 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:42.160 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:42.160 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:42.160 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:42.160 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:42.160 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:42.160 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:42.160 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:42.160 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:42.160 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:42.160 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:42.160 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:42.161 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:42.161 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:42.161 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:42.161 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:42.161 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:42.161 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:42.161 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:42.161 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:25:42.161 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:42.161 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:42.161 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:42.161 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:42.161 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:42.162 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:42.162 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:42.162 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:42.162 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:42.162 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:42.162 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:42.162 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:42.162 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:42.163 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:42.163 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:42.163 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:42.163 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:42.163 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:42.163 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:42.163 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:42.163 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:42.163 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:42.163 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:42.163 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:42.163 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:42.163 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:42.164 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:42.164 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:42.164 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:42.164 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:42.164 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:42.164 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:42.164 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:42.164 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:42.164 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:42.164 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:42.164 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:42.165 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:42.165 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:42.165 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:42.165 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:42.165 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:42.165 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:42.165 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:42.165 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:42.165 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:42.165 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:42.165 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:42.165 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:42.955 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0[107655]: 2024-03-24T11:25:42.637+0000 7efc569e37c0 -1 osd.0 0 read_superblock omap replica is missing. 2024-03-24T11:25:42.955 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0[107655]: 2024-03-24T11:25:42.670+0000 7efc569e37c0 -1 osd.0 108 log_to_monitors true 2024-03-24T11:25:42.956 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:42.956 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: pgmap v65: 161 pgs: 20 active+undersized+degraded, 38 active+undersized, 103 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 68/630 objects degraded (10.794%) 2024-03-24T11:25:42.956 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:42.956 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:42.956 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:42.956 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:42.956 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:42.956 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:42.956 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:42.956 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:42.956 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:42.956 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:42.956 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:42.957 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:42.957 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:42.957 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:42.957 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:42.957 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:42.957 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:42.957 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:42.957 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:42.957 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:42.957 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:42.957 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:42.957 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:42.957 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:42.958 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:42.958 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:42.958 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:42.958 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:42.958 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:42.958 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: pgmap v65: 161 pgs: 20 active+undersized+degraded, 38 active+undersized, 103 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 68/630 objects degraded (10.794%) 2024-03-24T11:25:42.958 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:42.958 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:42.958 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:42.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:42.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:42.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:42.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:42.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:42.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:42.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:42.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:42.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:42.960 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:42.960 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:42.960 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:42.960 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:42.960 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:42.960 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:42.960 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:42.960 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:42.960 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:42.960 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:42.960 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:42.960 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:42.960 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:42.961 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:42.961 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:42.961 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:42.961 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:42.961 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:42.961 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:42.961 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:42.961 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:42.961 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:42.962 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:42.962 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:42.962 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:42.962 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:42.962 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:42.962 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:42.962 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:42.962 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:42.962 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:42.962 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:42.962 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:42.962 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:42.962 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:42.963 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:42.963 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:42.963 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:42.963 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:42.963 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:42.963 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:42.963 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:42.963 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:42.963 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:42.963 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:42.963 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:42.964 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='osd.0 [v2:172.21.15.62:6802/2050191723,v1:172.21.15.62:6803/2050191723]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-24T11:25:42.964 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-24T11:25:42.964 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:42.964 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:42.964 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:42.964 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:42.964 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:42.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:42.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:42.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:42.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:42.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:42.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:42.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:42.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:42.967 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:42.967 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:42.967 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:42.967 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:42.967 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:42.967 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:42.967 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:42.967 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:42.967 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:42.967 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:42.967 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:42.967 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:42.967 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:42.967 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:42.968 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:42.968 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:42.968 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:42.968 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:42.968 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:42.968 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:42.968 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:42.968 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:42.968 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='osd.0 [v2:172.21.15.62:6802/2050191723,v1:172.21.15.62:6803/2050191723]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-24T11:25:42.968 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-24T11:25:42.968 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:42.968 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:42.968 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:42.968 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:42.968 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:43.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:43.400 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: pgmap v65: 161 pgs: 20 active+undersized+degraded, 38 active+undersized, 103 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 68/630 objects degraded (10.794%) 2024-03-24T11:25:43.400 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:43.400 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:43.400 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:43.400 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:43.400 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:43.400 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:43.401 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:43.401 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:43.401 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:43.401 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:43.401 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:43.401 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:43.401 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:43.401 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:43.401 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:43.402 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:43.402 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:43.402 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:43.402 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:43.402 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:43.402 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:43.403 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:43.403 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:43.403 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:43.403 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:43.403 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:43.403 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:43.403 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:43.403 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:43.403 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:43.404 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:43.404 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:43.404 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:43.405 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:43.405 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:43.405 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:43.406 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:43.406 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:43.406 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:43.406 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:43.407 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:43.407 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:43.407 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:43.407 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:43.407 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:43.407 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:43.408 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:43.408 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:43.408 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:43.408 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:43.408 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:43.408 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:43.408 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:43.408 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:43.408 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:43.409 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:43.409 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:43.409 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:43.409 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='osd.0 [v2:172.21.15.62:6802/2050191723,v1:172.21.15.62:6803/2050191723]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-24T11:25:43.409 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-24T11:25:43.409 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:43.409 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:43.410 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:43.410 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:43.410 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:43.789 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0[107655]: 2024-03-24T11:25:43.716+0000 7efc47e3b700 -1 osd.0 108 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T11:25:44.080 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:44.080 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:44.080 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:25:44.081 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:44.081 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:44.081 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:44.081 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:44.081 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:44.081 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:44.081 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:44.081 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:44.081 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:44.081 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:44.081 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:44.081 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:44.081 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:44.081 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:44.082 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:44.082 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:44.082 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:44.082 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:44.082 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:44.082 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:44.082 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:44.082 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:44.082 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:44.082 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:44.082 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:44.082 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:44.082 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:44.082 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:44.083 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:44.083 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:44.083 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:44.083 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:44.083 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:44.083 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:44.083 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:44.083 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:44.083 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:44.083 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:44.083 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:44.083 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:44.083 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:44.083 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:44.083 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:44.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-24T11:25:44.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: osdmap e111: 8 total, 7 up, 8 in 2024-03-24T11:25:44.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: from='osd.0 [v2:172.21.15.62:6802/2050191723,v1:172.21.15.62:6803/2050191723]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:25:44.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:25:44.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:44.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:44.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:44.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:44.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:44.085 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:44.085 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:44.085 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:25:44.085 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:44.085 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:44.085 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:44.085 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:44.085 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:44.085 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:44.085 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:44.085 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:44.085 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:44.085 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:44.085 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:44.086 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:44.086 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:44.086 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:44.086 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:44.086 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:44.086 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:44.086 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:44.088 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:44.088 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:44.088 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:44.088 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:44.088 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:44.088 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:44.088 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:44.088 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:44.088 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:44.088 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:44.088 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:44.088 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:44.089 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:44.089 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:44.089 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:44.089 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:44.089 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:44.089 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:44.089 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:44.089 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:44.089 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:44.089 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:44.089 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:44.089 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:44.089 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:44.089 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-24T11:25:44.089 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: osdmap e111: 8 total, 7 up, 8 in 2024-03-24T11:25:44.090 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: from='osd.0 [v2:172.21.15.62:6802/2050191723,v1:172.21.15.62:6803/2050191723]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:25:44.090 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:25:44.090 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:44.090 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:44.090 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:44.090 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:44.090 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:44.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:44.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:44.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:25:44.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:44.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:44.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:44.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:44.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:44.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:44.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:44.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:44.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:44.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:44.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:44.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:44.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:44.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:44.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:44.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:44.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:44.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:44.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:44.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:44.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:44.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:44.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:44.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:44.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:44.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:44.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:44.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:44.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:44.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:44.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:44.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:44.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:44.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:44.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:44.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:44.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:44.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:44.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:44.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:44.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:44.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:44.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:44.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-24T11:25:44.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: osdmap e111: 8 total, 7 up, 8 in 2024-03-24T11:25:44.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: from='osd.0 [v2:172.21.15.62:6802/2050191723,v1:172.21.15.62:6803/2050191723]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:25:44.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:25:44.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:44.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:44.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:44.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:44.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:44.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:44.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: pgmap v66: 161 pgs: 20 active+undersized+degraded, 38 active+undersized, 103 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 474 B/s rd, 0 op/s; 68/630 objects degraded (10.794%) 2024-03-24T11:25:44.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:44.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:44.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:44.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:44.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:44.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: failed to encode map e111 with expected crc 2024-03-24T11:25:44.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: failed to encode map e111 with expected crc 2024-03-24T11:25:44.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:44.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: failed to encode map e111 with expected crc 2024-03-24T11:25:44.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: failed to encode map e111 with expected crc 2024-03-24T11:25:44.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: failed to encode map e111 with expected crc 2024-03-24T11:25:44.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: failed to encode map e111 with expected crc 2024-03-24T11:25:44.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:44.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:44.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:44.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:44.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:44.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:44.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: failed to encode map e111 with expected crc 2024-03-24T11:25:44.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:44.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:44.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:44.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:44.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:44.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:44.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:44.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:44.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:44.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:44.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:44.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:44.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: pgmap v66: 161 pgs: 20 active+undersized+degraded, 38 active+undersized, 103 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 474 B/s rd, 0 op/s; 68/630 objects degraded (10.794%) 2024-03-24T11:25:44.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:44.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:44.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:44.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:44.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:44.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: failed to encode map e111 with expected crc 2024-03-24T11:25:44.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: failed to encode map e111 with expected crc 2024-03-24T11:25:44.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:44.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: failed to encode map e111 with expected crc 2024-03-24T11:25:44.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: failed to encode map e111 with expected crc 2024-03-24T11:25:44.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: failed to encode map e111 with expected crc 2024-03-24T11:25:44.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: failed to encode map e111 with expected crc 2024-03-24T11:25:44.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:44.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:44.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:44.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:44.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:44.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:44.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: failed to encode map e111 with expected crc 2024-03-24T11:25:44.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:44.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:44.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:44.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:44.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:44.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:44.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:44.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:44.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:44.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:44.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:44.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:44.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:44.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:44.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:44.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:44.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:44.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:44.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:44.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:44.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:44.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:44.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:44.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:44.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:44.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:44.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:44.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:44.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:44.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:44.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:44.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:44.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:44.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:44.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:44.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T11:25:44.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: osd.0 [v2:172.21.15.62:6802/2050191723,v1:172.21.15.62:6803/2050191723] boot 2024-03-24T11:25:44.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: osdmap e112: 8 total, 8 up, 8 in 2024-03-24T11:25:44.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T11:25:44.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:44.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:44.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:44.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:44.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:44.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:44.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:44.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:44.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:44.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:44.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:44.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:44.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:44.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:44.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:44.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:44.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:44.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:44.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:44.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:44.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:44.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:44.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:44.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:44.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:44.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:44.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:44.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:44.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:44.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T11:25:44.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: osd.0 [v2:172.21.15.62:6802/2050191723,v1:172.21.15.62:6803/2050191723] boot 2024-03-24T11:25:44.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: osdmap e112: 8 total, 8 up, 8 in 2024-03-24T11:25:44.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T11:25:44.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:44.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:44.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:44.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:44.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:45.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:45.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: pgmap v66: 161 pgs: 20 active+undersized+degraded, 38 active+undersized, 103 active+clean; 585 KiB data, 97 MiB used, 715 GiB / 715 GiB avail; 474 B/s rd, 0 op/s; 68/630 objects degraded (10.794%) 2024-03-24T11:25:45.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:45.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:45.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:45.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:45.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:45.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: failed to encode map e111 with expected crc 2024-03-24T11:25:45.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: failed to encode map e111 with expected crc 2024-03-24T11:25:45.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:45.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: failed to encode map e111 with expected crc 2024-03-24T11:25:45.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: failed to encode map e111 with expected crc 2024-03-24T11:25:45.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: failed to encode map e111 with expected crc 2024-03-24T11:25:45.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: failed to encode map e111 with expected crc 2024-03-24T11:25:45.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:45.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:45.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:45.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:45.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:45.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:45.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: failed to encode map e111 with expected crc 2024-03-24T11:25:45.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:45.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:45.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:45.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:45.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:45.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:45.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:45.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:45.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:45.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:45.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:45.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:45.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:45.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:45.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:45.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:45.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:45.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:45.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:45.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:45.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:45.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:45.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:45.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:45.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:45.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:45.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:45.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:45.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:45.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:45.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:45.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:45.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:45.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:45.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:45.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T11:25:45.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: osd.0 [v2:172.21.15.62:6802/2050191723,v1:172.21.15.62:6803/2050191723] boot 2024-03-24T11:25:45.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: osdmap e112: 8 total, 8 up, 8 in 2024-03-24T11:25:45.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T11:25:45.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:45.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:45.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:45.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:45.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:45.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:45.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:45.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:45.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:45.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:45.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:45.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:45.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: failed to encode map e112 with expected crc 2024-03-24T11:25:45.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: failed to encode map e112 with expected crc 2024-03-24T11:25:45.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: failed to encode map e112 with expected crc 2024-03-24T11:25:45.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: failed to encode map e112 with expected crc 2024-03-24T11:25:45.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:45.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:45.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:45.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:45.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:45.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:45.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:45.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:45.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:45.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:45.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:45.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:45.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:45.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:45.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:45.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:45.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:45.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:45.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:45.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:45.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:45.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:45.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:45.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:45.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:45.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:45.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:45.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:45.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: failed to encode map e112 with expected crc 2024-03-24T11:25:45.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: failed to encode map e112 with expected crc 2024-03-24T11:25:45.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: failed to encode map e112 with expected crc 2024-03-24T11:25:45.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: failed to encode map e112 with expected crc 2024-03-24T11:25:45.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:45.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:45.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:45.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:45.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:45.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:45.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:45.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:45.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:45.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:45.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:45.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:45.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:45.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:45.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:45.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:45.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:45.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:45.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:45.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:45.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:45.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:45.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:45.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:45.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:45.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:45.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:45.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:45.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:45.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:45.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:45.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:45.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:45.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:45.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:45.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:45.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:45.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:45.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:45.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:45.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:45.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:45.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:45.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:45.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:45.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:45.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:45.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:45.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: osdmap e113: 8 total, 8 up, 8 in 2024-03-24T11:25:45.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:45.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:45.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:45.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:45.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:45.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:45.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:45.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:45.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:45.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:45.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:45.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:45.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:45.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:45.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:45.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:45.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:45.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:45.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:45.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:45.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:45.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:45.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:45.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:45.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:45.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:45.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:45.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:45.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:45.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:45.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:45.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:45.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: osdmap e113: 8 total, 8 up, 8 in 2024-03-24T11:25:45.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:45.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:45.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:45.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:45.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:46.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:46.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:46.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:46.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:46.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:46.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:46.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:46.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: failed to encode map e112 with expected crc 2024-03-24T11:25:46.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: failed to encode map e112 with expected crc 2024-03-24T11:25:46.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: failed to encode map e112 with expected crc 2024-03-24T11:25:46.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: failed to encode map e112 with expected crc 2024-03-24T11:25:46.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:46.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:46.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:46.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:46.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:46.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:46.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:46.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:46.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:46.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:46.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:46.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:46.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:46.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:46.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:46.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:46.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:46.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:46.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:46.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:46.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:46.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:46.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:46.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:46.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:46.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:46.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:46.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:46.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:46.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:46.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:46.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:46.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:46.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:46.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:46.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:46.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:46.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:46.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:46.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:46.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:46.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:46.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:46.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:46.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:46.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:46.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:46.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:46.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: osdmap e113: 8 total, 8 up, 8 in 2024-03-24T11:25:46.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:46.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:46.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:46.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:46.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:47.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:47.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: pgmap v69: 161 pgs: 20 active+undersized+degraded, 38 active+undersized, 103 active+clean; 585 KiB data, 113 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 68/630 objects degraded (10.794%) 2024-03-24T11:25:47.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:47.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:47.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:47.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:47.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:47.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:47.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: failed to encode map e113 with expected crc 2024-03-24T11:25:47.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: failed to encode map e113 with expected crc 2024-03-24T11:25:47.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: failed to encode map e113 with expected crc 2024-03-24T11:25:47.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: failed to encode map e113 with expected crc 2024-03-24T11:25:47.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: failed to encode map e113 with expected crc 2024-03-24T11:25:47.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: failed to encode map e113 with expected crc 2024-03-24T11:25:47.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: failed to encode map e113 with expected crc 2024-03-24T11:25:47.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: failed to encode map e113 with expected crc 2024-03-24T11:25:47.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: failed to encode map e113 with expected crc 2024-03-24T11:25:47.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: failed to encode map e113 with expected crc 2024-03-24T11:25:47.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: failed to encode map e113 with expected crc 2024-03-24T11:25:47.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:47.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:47.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:47.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:47.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:47.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:47.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:47.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:47.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:47.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:47.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:47.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:47.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:47.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:47.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:47.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:47.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:47.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:47.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:47.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:47.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:47.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:47.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:47.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:47.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:47.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:47.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:47.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:47.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:47.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:47.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:47.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:47.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:47.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:47.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:47.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:47.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:47.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:47.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:47.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:47.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:47.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:47.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:47.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:47.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:47.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:47.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:47.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:47.113 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:47.113 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: pgmap v69: 161 pgs: 20 active+undersized+degraded, 38 active+undersized, 103 active+clean; 585 KiB data, 113 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 68/630 objects degraded (10.794%) 2024-03-24T11:25:47.113 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:47.114 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:47.114 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:47.114 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:47.114 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:47.114 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:47.114 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: failed to encode map e113 with expected crc 2024-03-24T11:25:47.114 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: failed to encode map e113 with expected crc 2024-03-24T11:25:47.114 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: failed to encode map e113 with expected crc 2024-03-24T11:25:47.114 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: failed to encode map e113 with expected crc 2024-03-24T11:25:47.114 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: failed to encode map e113 with expected crc 2024-03-24T11:25:47.114 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: failed to encode map e113 with expected crc 2024-03-24T11:25:47.114 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: failed to encode map e113 with expected crc 2024-03-24T11:25:47.114 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: failed to encode map e113 with expected crc 2024-03-24T11:25:47.114 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: failed to encode map e113 with expected crc 2024-03-24T11:25:47.114 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: failed to encode map e113 with expected crc 2024-03-24T11:25:47.115 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: failed to encode map e113 with expected crc 2024-03-24T11:25:47.115 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:47.115 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:47.115 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:47.115 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:47.115 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:47.115 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:47.115 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:47.115 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:47.115 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:47.115 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:47.115 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:47.115 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:47.115 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:47.115 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:47.115 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:47.116 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:47.116 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: pgmap v69: 161 pgs: 20 active+undersized+degraded, 38 active+undersized, 103 active+clean; 585 KiB data, 113 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 68/630 objects degraded (10.794%) 2024-03-24T11:25:47.116 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:47.116 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:47.116 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:47.116 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:47.116 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:47.116 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:47.116 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: failed to encode map e113 with expected crc 2024-03-24T11:25:47.116 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: failed to encode map e113 with expected crc 2024-03-24T11:25:47.116 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: failed to encode map e113 with expected crc 2024-03-24T11:25:47.117 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: failed to encode map e113 with expected crc 2024-03-24T11:25:47.117 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: failed to encode map e113 with expected crc 2024-03-24T11:25:47.117 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: failed to encode map e113 with expected crc 2024-03-24T11:25:47.117 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: failed to encode map e113 with expected crc 2024-03-24T11:25:47.117 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: failed to encode map e113 with expected crc 2024-03-24T11:25:47.117 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: failed to encode map e113 with expected crc 2024-03-24T11:25:47.117 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: failed to encode map e113 with expected crc 2024-03-24T11:25:47.117 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: failed to encode map e113 with expected crc 2024-03-24T11:25:47.117 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:47.117 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:47.117 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:47.117 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:47.117 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:47.117 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:47.117 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:47.118 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:47.118 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:47.118 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:47.118 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:47.118 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:47.118 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:47.118 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:47.118 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:47.118 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:47.118 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:47.118 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:47.119 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:47.119 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:47.119 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:47.119 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:47.119 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:47.119 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:47.119 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:47.119 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:47.119 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:47.119 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:47.119 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:47.119 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:47.119 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:47.119 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:47.119 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:47.119 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:47.120 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:47.120 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:47.120 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:47.120 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:47.120 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:47.120 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:47.120 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:47.120 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:47.120 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:47.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:47.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:47.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:47.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:47.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:47.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:47.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:47.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:47.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:47.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:47.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:47.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:47.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:47.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:47.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:47.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:47.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:47.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:47.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:47.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:47.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:47.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:47.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:47.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:47.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:47.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:47.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:47.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:47.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:47.122 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:47.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:47.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:47.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:47.123 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:47.123 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:47.123 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:47.123 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:47.123 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:47.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:47.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:47.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:47.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:47.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:47.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:47.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:47.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:47.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:47.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:47.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:47.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:47.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:47.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:47.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:47.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:47.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:47.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:47.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:47.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:47.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:47.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:47.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:47.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:47.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:47.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:47.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:47.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:47.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:47.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:47.994 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:47.995 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:47.995 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:47.995 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:47.995 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:47.995 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:47.995 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:47.995 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:47.995 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:47.995 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:47.995 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:47.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:47.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:47.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:47.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:47.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:47.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:47.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:47.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:47.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:47.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:47.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:47.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:47.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:47.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:47.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:47.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:47.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:47.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:47.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:47.998 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:47.998 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:47.998 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:47.998 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:47.998 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:47.998 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:47.999 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:47.999 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:47.999 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:47.999 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:47.999 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:47.999 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:47.999 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:47.999 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:48.000 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:48.000 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:48.000 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:48.000 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:48.000 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:48.000 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:48.000 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:48.001 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:48.001 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:48.001 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:48.001 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:48.001 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:48.002 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:48.002 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:48.002 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:48.002 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:48.002 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:48.002 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:48.002 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:48.002 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:48.003 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:48.003 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:48.003 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:48.003 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:48.003 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:48.003 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:48.004 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:48.004 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:48.004 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:48.005 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:48.005 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:48.005 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:48.005 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:48.006 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:48.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:48.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:48.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:48.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:48.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:48.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:48.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:48.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:48.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:48.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:48.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:48.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:48.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:48.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:48.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:48.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:48.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:48.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:48.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:48.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:48.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:48.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:48.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:48.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:48.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:48.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:48.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:48.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:48.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:48.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:48.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:48.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:48.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:48.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:48.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:48.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:48.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:48.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:48.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:48.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:48.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:48.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:48.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:48.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:48.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:48.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:48.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:48.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:48.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:48.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:48.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:48.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:48.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:48.115 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.108 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:25:48.659Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"smithi062:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"172.21.15.139:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-24T11:25:49.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: pgmap v71: 161 pgs: 7 active+undersized+degraded, 14 active+undersized, 140 active+clean; 585 KiB data, 114 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s; 28/630 objects degraded (4.444%) 2024-03-24T11:25:49.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: Health check update: Degraded data redundancy: 28/630 objects degraded (4.444%), 7 pgs degraded (PG_DEGRADED) 2024-03-24T11:25:49.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.115 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.115 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.115 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.115 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.115 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.115 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.115 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.115 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.115 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.115 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.115 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.116 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.116 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.116 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.116 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.117 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.117 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.117 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.117 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.117 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.118 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.118 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.118 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.118 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.118 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.118 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.118 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.118 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.118 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.118 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.157 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: pgmap v71: 161 pgs: 7 active+undersized+degraded, 14 active+undersized, 140 active+clean; 585 KiB data, 114 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s; 28/630 objects degraded (4.444%) 2024-03-24T11:25:49.157 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.157 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.157 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.157 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.158 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.158 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.158 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.158 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.158 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.158 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.158 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.158 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.158 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.158 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.158 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.158 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.158 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.158 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.159 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.159 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: Health check update: Degraded data redundancy: 28/630 objects degraded (4.444%), 7 pgs degraded (PG_DEGRADED) 2024-03-24T11:25:49.159 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.159 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.159 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.159 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.159 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.159 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.159 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.159 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.159 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: pgmap v71: 161 pgs: 7 active+undersized+degraded, 14 active+undersized, 140 active+clean; 585 KiB data, 114 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s; 28/630 objects degraded (4.444%) 2024-03-24T11:25:49.160 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.160 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.160 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.160 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.160 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.160 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.160 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.160 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.160 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.160 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.160 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.160 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.160 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.160 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.161 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.161 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.161 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.161 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.161 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.161 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: Health check update: Degraded data redundancy: 28/630 objects degraded (4.444%), 7 pgs degraded (PG_DEGRADED) 2024-03-24T11:25:49.161 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.161 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.161 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.161 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.161 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.161 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.161 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.161 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.162 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.162 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.162 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.162 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.162 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.162 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.162 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.162 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.163 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.163 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.163 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.163 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.163 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.163 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.163 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.163 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.163 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.163 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.163 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.163 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.163 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.163 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.163 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.163 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.164 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.164 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.164 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.164 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.164 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.164 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.164 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.164 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.164 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.164 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.164 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.164 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.164 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.165 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.165 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.165 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.165 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.165 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.165 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.165 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.166 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.166 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.166 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.166 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.166 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.166 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.166 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.166 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.166 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.166 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.166 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.166 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.166 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.166 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.166 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.167 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.167 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.167 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.167 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.167 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.167 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.167 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.167 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.167 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.167 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.167 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.167 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.167 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.168 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.168 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.168 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.168 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:49.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:49.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:49.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:49.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:49.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:49.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:50.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:50.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:50.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:50.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:50.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:50.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:50.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:50.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:50.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:50.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:50.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:50.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:50.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:50.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:50.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:50.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:50.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:50.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:50.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:50.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:50.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:50.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:50.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:50.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:50.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:50.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:50.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:50.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:50.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:50.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:50.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:50.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:50.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:50.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:50.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:50.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:50.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:50.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:50.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:50.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:50.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:50.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:50.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:50.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:50.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:50.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:50.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:50.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:50.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:50.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:50.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:50.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:50.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:50.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:50.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:50.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:50.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:50.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:50.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:50.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:50.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:50.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:50.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:50.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:50.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:50.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:51.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:50 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:51.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:50 smithi139 ceph-mon[91894]: pgmap v72: 161 pgs: 161 active+clean; 585 KiB data, 114 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:25:51.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:50 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:51.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:50 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:51.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:50 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:51.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:50 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:51.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:50 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:51.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:50 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:51.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:50 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:51.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:50 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:51.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:50 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:51.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:50 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:51.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:50 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:51.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:50 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:51.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:50 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:51.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:50 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:51.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:50 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:51.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:50 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:51.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:50 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:51.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:50 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:51.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:50 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:51.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:50 smithi139 ceph-mon[91894]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 28/630 objects degraded (4.444%), 7 pgs degraded) 2024-03-24T11:25:51.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:50 smithi139 ceph-mon[91894]: Cluster is now healthy 2024-03-24T11:25:51.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:50 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:51.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:50 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:51.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:50 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:51.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:50 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:51.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:50 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:51.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:50 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:51.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:50 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:51.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:50 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:51.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:50 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:51.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:50 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:51.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:50 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:51.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:50 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:51.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:50 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-24T11:25:51.156 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:25:50] "GET /metrics HTTP/1.1" 200 38472 "" "Prometheus/2.33.4" 2024-03-24T11:25:51.156 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:51.157 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[90553]: pgmap v72: 161 pgs: 161 active+clean; 585 KiB data, 114 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:25:51.157 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:51.157 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:51.157 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:51.157 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:51.157 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:51.157 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:51.157 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:51.157 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:51.157 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:51.157 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:51.157 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:51.157 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:51.157 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:51.157 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:51.158 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:51.158 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:51.158 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:51.158 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:51.158 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:51.158 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[90553]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 28/630 objects degraded (4.444%), 7 pgs degraded) 2024-03-24T11:25:51.158 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[90553]: Cluster is now healthy 2024-03-24T11:25:51.158 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:51.158 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:51.158 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:51.158 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:51.158 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:51.158 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:51.158 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:51.158 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:51.159 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:51.159 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:51.159 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:51.159 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:51.159 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-24T11:25:51.159 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:51.159 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[98643]: pgmap v72: 161 pgs: 161 active+clean; 585 KiB data, 114 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:25:51.159 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:51.159 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:51.159 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:51.160 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:51.160 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:51.160 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:51.160 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:51.160 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:51.160 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:51.160 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:51.160 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:51.160 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:51.160 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:51.161 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:51.161 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:51.161 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:51.161 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:51.161 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:51.161 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:51.161 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[98643]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 28/630 objects degraded (4.444%), 7 pgs degraded) 2024-03-24T11:25:51.161 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[98643]: Cluster is now healthy 2024-03-24T11:25:51.161 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:51.161 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:51.161 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:51.161 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:25:51.161 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:51.161 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:51.161 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:51.162 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:51.162 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:25:51.162 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:51.162 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:51.162 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:51.162 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:50 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-24T11:25:51.441 INFO:teuthology.orchestra.run.smithi062.stdout:true 2024-03-24T11:25:52.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:51 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:52.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:51 smithi139 ceph-mon[91894]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-24T11:25:52.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:51 smithi139 ceph-mon[91894]: Upgrade: osd.0 is safe to restart 2024-03-24T11:25:52.122 INFO:teuthology.orchestra.run.smithi062.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T11:25:52.123 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi062 smithi062 running 1s ago 15m - - 2024-03-24T11:25:52.123 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi139 smithi139 running 13s ago 15m - - 2024-03-24T11:25:52.123 INFO:teuthology.orchestra.run.smithi062.stdout:alertmanager.a smithi062 *:9093,9094 running (5m) 1s ago 9m 24.4M - 0.23.0 ba2b418f427c 818298d58140 2024-03-24T11:25:52.123 INFO:teuthology.orchestra.run.smithi062.stdout:grafana.a smithi139 *:3000 running (4m) 13s ago 9m 53.9M - 8.3.5 dad864ee21e9 b48167e7171e 2024-03-24T11:25:52.123 INFO:teuthology.orchestra.run.smithi062.stdout:iscsi.foo.smithi062.dggsvk smithi062 running (3m) 1s ago 8m 56.6M - 3.5 e1d6a67b021e 0a98806ec839 2024-03-24T11:25:52.123 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.x smithi139 *:8443,9283,8765 running (2m) 13s ago 14m 449M - 19.0.0-1667-gdb0330b1 4f325f8440e9 d94987b11387 2024-03-24T11:25:52.123 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.y smithi062 *:8443,9283,8765 running (3m) 1s ago 16m 512M - 19.0.0-1667-gdb0330b1 4f325f8440e9 ca1a49028d65 2024-03-24T11:25:52.123 INFO:teuthology.orchestra.run.smithi062.stdout:mon.a smithi062 running (113s) 1s ago 16m 49.3M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 f7afe6e28c88 2024-03-24T11:25:52.123 INFO:teuthology.orchestra.run.smithi062.stdout:mon.b smithi139 running (58s) 13s ago 14m 33.1M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 ca357beea4dd 2024-03-24T11:25:52.123 INFO:teuthology.orchestra.run.smithi062.stdout:mon.c smithi062 running (79s) 1s ago 14m 42.6M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 459e7449ec39 2024-03-24T11:25:52.123 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.a smithi062 *:9100 running (5m) 1s ago 10m 17.0M - 1.3.1 1dbe0e931976 79d5a02a999f 2024-03-24T11:25:52.123 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.b smithi139 *:9100 running (4m) 13s ago 10m 15.7M - 1.3.1 1dbe0e931976 8f9f4608e465 2024-03-24T11:25:52.123 INFO:teuthology.orchestra.run.smithi062.stdout:osd.0 smithi062 running (14m) 1s ago 14m 54.1M 4096M 17.2.0 e1d6a67b021e 8040b30d7170 2024-03-24T11:25:52.124 INFO:teuthology.orchestra.run.smithi062.stdout:osd.1 smithi062 running (13m) 1s ago 13m 55.7M 4096M 17.2.0 e1d6a67b021e e62d22e5dc29 2024-03-24T11:25:52.124 INFO:teuthology.orchestra.run.smithi062.stdout:osd.2 smithi062 running (13m) 1s ago 13m 49.2M 4096M 17.2.0 e1d6a67b021e 714f550103a7 2024-03-24T11:25:52.124 INFO:teuthology.orchestra.run.smithi062.stdout:osd.3 smithi062 running (13m) 1s ago 13m 53.8M 4096M 17.2.0 e1d6a67b021e 1d2868bd5195 2024-03-24T11:25:52.124 INFO:teuthology.orchestra.run.smithi062.stdout:osd.4 smithi139 running (12m) 13s ago 12m 58.7M 4096M 17.2.0 e1d6a67b021e 7568bfa9e74d 2024-03-24T11:25:52.124 INFO:teuthology.orchestra.run.smithi062.stdout:osd.5 smithi139 running (12m) 13s ago 12m 52.2M 4096M 17.2.0 e1d6a67b021e dd0abd599a14 2024-03-24T11:25:52.124 INFO:teuthology.orchestra.run.smithi062.stdout:osd.6 smithi139 running (11m) 13s ago 11m 51.9M 4096M 17.2.0 e1d6a67b021e a4b1289aca69 2024-03-24T11:25:52.124 INFO:teuthology.orchestra.run.smithi062.stdout:osd.7 smithi139 running (11m) 13s ago 11m 54.6M 4096M 17.2.0 e1d6a67b021e bc7fadbc1384 2024-03-24T11:25:52.124 INFO:teuthology.orchestra.run.smithi062.stdout:prometheus.a smithi139 *:9095 running (3m) 13s ago 10m 71.0M - 2.33.4 514e6a882f6e 2ed47f810d33 2024-03-24T11:25:52.124 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi062.sacpds smithi062 *:8000 running (9m) 1s ago 9m 73.2M - 17.2.0 e1d6a67b021e 89f4ac308e98 2024-03-24T11:25:52.124 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi139.yljesh smithi139 *:8000 running (9m) 13s ago 9m 72.2M - 17.2.0 e1d6a67b021e e189bdc12cae 2024-03-24T11:25:52.124 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi062.xvpxnk smithi062 *:80 running (8m) 1s ago 8m 73.1M - 17.2.0 e1d6a67b021e d1aa74adab44 2024-03-24T11:25:52.124 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi139.pbtrmp smithi139 *:80 running (8m) 13s ago 8m 75.7M - 17.2.0 e1d6a67b021e 7f8e336755d7 2024-03-24T11:25:52.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:51 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:52.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:51 smithi062 ceph-mon[90553]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-24T11:25:52.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:51 smithi062 ceph-mon[90553]: Upgrade: osd.0 is safe to restart 2024-03-24T11:25:52.125 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:51 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:25:52.125 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:51 smithi062 ceph-mon[98643]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-24T11:25:52.125 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:51 smithi062 ceph-mon[98643]: Upgrade: osd.0 is safe to restart 2024-03-24T11:25:52.554 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:25:52.554 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": { 2024-03-24T11:25:52.554 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 3 2024-03-24T11:25:52.555 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:25:52.555 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": { 2024-03-24T11:25:52.555 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-24T11:25:52.555 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:25:52.555 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": { 2024-03-24T11:25:52.555 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 7, 2024-03-24T11:25:52.555 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 1 2024-03-24T11:25:52.555 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:25:52.555 INFO:teuthology.orchestra.run.smithi062.stdout: "rgw": { 2024-03-24T11:25:52.555 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4 2024-03-24T11:25:52.555 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:25:52.555 INFO:teuthology.orchestra.run.smithi062.stdout: "overall": { 2024-03-24T11:25:52.555 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 11, 2024-03-24T11:25:52.555 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 6 2024-03-24T11:25:52.555 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-24T11:25:52.556 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:25:52.902 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:25:52.902 INFO:teuthology.orchestra.run.smithi062.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69", 2024-03-24T11:25:52.902 INFO:teuthology.orchestra.run.smithi062.stdout: "in_progress": true, 2024-03-24T11:25:52.902 INFO:teuthology.orchestra.run.smithi062.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T11:25:52.902 INFO:teuthology.orchestra.run.smithi062.stdout: "services_complete": [ 2024-03-24T11:25:52.902 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr", 2024-03-24T11:25:52.902 INFO:teuthology.orchestra.run.smithi062.stdout: "mon" 2024-03-24T11:25:52.902 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-03-24T11:25:52.902 INFO:teuthology.orchestra.run.smithi062.stdout: "progress": "5/23 daemons upgraded", 2024-03-24T11:25:52.902 INFO:teuthology.orchestra.run.smithi062.stdout: "message": "Currently upgrading osd daemons", 2024-03-24T11:25:52.903 INFO:teuthology.orchestra.run.smithi062.stdout: "is_paused": false 2024-03-24T11:25:52.903 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:25:53.068 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:52 smithi139 ceph-mon[91894]: pgmap v73: 161 pgs: 161 active+clean; 585 KiB data, 114 MiB used, 715 GiB / 715 GiB avail; 690 B/s rd, 0 op/s 2024-03-24T11:25:53.068 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:52 smithi139 ceph-mon[91894]: from='client.64180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:25:53.068 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:52 smithi139 ceph-mon[91894]: from='client.44220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:25:53.068 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:52 smithi139 ceph-mon[91894]: from='client.? 172.21.15.62:0/3557616035' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:53.143 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:52 smithi062 ceph-mon[90553]: pgmap v73: 161 pgs: 161 active+clean; 585 KiB data, 114 MiB used, 715 GiB / 715 GiB avail; 690 B/s rd, 0 op/s 2024-03-24T11:25:53.143 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:52 smithi062 ceph-mon[90553]: from='client.64180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:25:53.143 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:52 smithi062 ceph-mon[90553]: from='client.44220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:25:53.143 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:52 smithi062 ceph-mon[90553]: from='client.? 172.21.15.62:0/3557616035' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:53.143 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:52 smithi062 ceph-mon[98643]: pgmap v73: 161 pgs: 161 active+clean; 585 KiB data, 114 MiB used, 715 GiB / 715 GiB avail; 690 B/s rd, 0 op/s 2024-03-24T11:25:53.143 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:52 smithi062 ceph-mon[98643]: from='client.64180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:25:53.144 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:52 smithi062 ceph-mon[98643]: from='client.44220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:25:53.144 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:52 smithi062 ceph-mon[98643]: from='client.? 172.21.15.62:0/3557616035' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:25:53.318 INFO:teuthology.orchestra.run.smithi062.stdout:HEALTH_OK 2024-03-24T11:25:54.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:53 smithi139 ceph-mon[91894]: from='client.64192 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:25:54.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:53 smithi139 ceph-mon[91894]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:25:54.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:53 smithi139 ceph-mon[91894]: from='client.44238 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:25:54.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:53 smithi139 ceph-mon[91894]: from='client.? 172.21.15.62:0/354894546' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:25:54.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:53 smithi062 ceph-mon[98643]: from='client.64192 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:25:54.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:53 smithi062 ceph-mon[98643]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:25:54.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:53 smithi062 ceph-mon[98643]: from='client.44238 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:25:54.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:53 smithi062 ceph-mon[98643]: from='client.? 172.21.15.62:0/354894546' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:25:54.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:53 smithi062 ceph-mon[90553]: from='client.64192 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:25:54.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:53 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:25:54.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:53 smithi062 ceph-mon[90553]: from='client.44238 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:25:54.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:53 smithi062 ceph-mon[90553]: from='client.? 172.21.15.62:0/354894546' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:25:54.960 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:54 smithi139 ceph-mon[91894]: pgmap v74: 161 pgs: 161 active+clean; 585 KiB data, 114 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:25:54.960 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:54 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-24T11:25:54.960 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:54 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:54.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:54 smithi062 ceph-mon[90553]: pgmap v74: 161 pgs: 161 active+clean; 585 KiB data, 114 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:25:54.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:54 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-24T11:25:54.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:54 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:54.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:54 smithi062 ceph-mon[98643]: pgmap v74: 161 pgs: 161 active+clean; 585 KiB data, 114 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:25:54.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:54 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-24T11:25:54.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:54 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:25:55.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:55 smithi062 ceph-mon[90553]: Upgrade: Updating osd.0 2024-03-24T11:25:55.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:55 smithi062 ceph-mon[90553]: Deploying daemon osd.0 on smithi062 2024-03-24T11:25:55.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:55 smithi062 ceph-mon[98643]: Upgrade: Updating osd.0 2024-03-24T11:25:55.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:55 smithi062 ceph-mon[98643]: Deploying daemon osd.0 on smithi062 2024-03-24T11:25:56.009 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:55 smithi139 ceph-mon[91894]: Upgrade: Updating osd.0 2024-03-24T11:25:56.009 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:55 smithi139 ceph-mon[91894]: Deploying daemon osd.0 on smithi062 2024-03-24T11:25:57.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:56 smithi139 ceph-mon[91894]: pgmap v75: 161 pgs: 161 active+clean; 585 KiB data, 114 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T11:25:57.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:57.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:25:57.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:57.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:56 smithi062 ceph-mon[90553]: pgmap v75: 161 pgs: 161 active+clean; 585 KiB data, 114 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T11:25:57.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:57.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:25:57.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:57.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:56 smithi062 ceph-mon[98643]: pgmap v75: 161 pgs: 161 active+clean; 585 KiB data, 114 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T11:25:57.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:57.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:25:57.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:58.181 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:58.181 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:58.181 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:58.181 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:58.181 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:58.181 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:58.182 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:58.182 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:58.182 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:58.182 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:58.182 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:58.182 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:58.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:58.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:58.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:58.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:58.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:58.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:58.919 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:25:58 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:25:58.659Z caller=alerting.go:353 level=warn component="rule manager" alert="unsupported value type" msg="Expanding alert template failed" err="error executing template __alert_CephOSDDown: template: __alert_CephOSDDown:1:358: executing \"__alert_CephOSDDown\" at : error calling query: found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"smithi062:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"172.21.15.139:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}];many-to-many matching not allowed: matching labels must be unique on one side" data="unsupported value type" 2024-03-24T11:25:58.919 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:25:58 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:25:58.660Z caller=manager.go:609 level=warn component="rule manager" group=osd msg="Evaluating rule failed" rule="alert: CephOSDFlapping\nexpr: (rate(ceph_osd_up[5m]) * on(ceph_daemon) group_left(hostname) ceph_osd_metadata)\n * 60 > 1\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.4.4\n severity: warning\n type: ceph_default\nannotations:\n description: OSD {{ $labels.ceph_daemon }} on {{ $labels.hostname }} was marked\n down and back up {{ $value | humanize }} times once a minute for 5 minutes. This\n may indicate a network issue (latency, packet loss, MTU mismatch) on the cluster\n network, or the public network if no cluster network is deployed. Check the network\n stats on the listed host(s).\n documentation: https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-osd#flapping-osds\n summary: Network issues are causing OSDs to flap (mark each other down)\n" err="found duplicate series for the match group {ceph_daemon=\"osd.0\"} on the right hand-side of the operation: [{__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"smithi062:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}, {__name__=\"ceph_osd_metadata\", ceph_daemon=\"osd.0\", ceph_version=\"ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)\", cluster_addr=\"172.21.15.62\", device_class=\"ssd\", hostname=\"smithi062\", instance=\"172.21.15.139:9283\", job=\"ceph\", objectstore=\"bluestore\", public_addr=\"172.21.15.62\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-24T11:25:58.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:58 smithi062 ceph-mon[90553]: pgmap v76: 161 pgs: 161 active+clean; 585 KiB data, 114 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T11:25:58.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:58.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:58.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:58.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:58 smithi062 ceph-mon[98643]: pgmap v76: 161 pgs: 161 active+clean; 585 KiB data, 114 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T11:25:58.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:58.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:58.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:59.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:58 smithi139 ceph-mon[91894]: pgmap v76: 161 pgs: 161 active+clean; 585 KiB data, 114 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T11:25:59.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:59.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:59.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:25:59.727 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:59 smithi062 systemd[1]: Stopping Ceph osd.0 for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:26:00.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:25:59 smithi062 ceph-mon[90553]: osd.0 marked itself down and dead 2024-03-24T11:26:00.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:25:59 smithi062 ceph-mon[98643]: osd.0 marked itself down and dead 2024-03-24T11:26:00.227 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:59 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0[107655]: 2024-03-24T11:25:59.864+0000 7efc4d04a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T11:26:00.227 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:59 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0[107655]: 2024-03-24T11:25:59.864+0000 7efc4d04a700 -1 osd.0 113 *** Got signal Terminated *** 2024-03-24T11:26:00.227 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:25:59 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0[107655]: 2024-03-24T11:25:59.865+0000 7efc4d04a700 -1 osd.0 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T11:26:00.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:25:59 smithi139 ceph-mon[91894]: osd.0 marked itself down and dead 2024-03-24T11:26:00.727 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:26:00 smithi062 podman[113123]: 2024-03-24 11:26:00.425880955 +0000 UTC m=+0.693212188 container died 61e78e75257c59caf926fe47afc07a9a349c3e5d30dfbeba6180bef63b5c0307 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-db0330b, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-03-24T11:26:01.046 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:26:00 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:26:00] "GET /metrics HTTP/1.1" 200 38495 "" "Prometheus/2.33.4" 2024-03-24T11:26:01.047 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:26:00 smithi062 podman[113123]: 2024-03-24 11:26:00.786745066 +0000 UTC m=+1.054076308 container cleanup 61e78e75257c59caf926fe47afc07a9a349c3e5d30dfbeba6180bef63b5c0307 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T11:26:01.047 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:26:00 smithi062 bash[113123]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0 2024-03-24T11:26:01.047 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:00 smithi062 ceph-mon[98643]: pgmap v77: 161 pgs: 161 active+clean; 585 KiB data, 114 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:26:01.047 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:00 smithi062 ceph-mon[98643]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T11:26:01.047 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:00 smithi062 ceph-mon[98643]: failed to encode map e114 with expected crc 2024-03-24T11:26:01.048 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:00 smithi062 ceph-mon[98643]: failed to encode map e114 with expected crc 2024-03-24T11:26:01.048 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:00 smithi062 ceph-mon[98643]: osdmap e114: 8 total, 7 up, 8 in 2024-03-24T11:26:01.048 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:00 smithi062 ceph-mon[98643]: failed to encode map e114 with expected crc 2024-03-24T11:26:01.048 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:00 smithi062 ceph-mon[98643]: failed to encode map e114 with expected crc 2024-03-24T11:26:01.048 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:00 smithi062 ceph-mon[98643]: failed to encode map e114 with expected crc 2024-03-24T11:26:01.048 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:00 smithi062 ceph-mon[90553]: pgmap v77: 161 pgs: 161 active+clean; 585 KiB data, 114 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:26:01.048 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:00 smithi062 ceph-mon[90553]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T11:26:01.048 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:00 smithi062 ceph-mon[90553]: failed to encode map e114 with expected crc 2024-03-24T11:26:01.048 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:00 smithi062 ceph-mon[90553]: failed to encode map e114 with expected crc 2024-03-24T11:26:01.048 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:00 smithi062 ceph-mon[90553]: osdmap e114: 8 total, 7 up, 8 in 2024-03-24T11:26:01.048 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:00 smithi062 ceph-mon[90553]: failed to encode map e114 with expected crc 2024-03-24T11:26:01.048 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:00 smithi062 ceph-mon[90553]: failed to encode map e114 with expected crc 2024-03-24T11:26:01.049 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:00 smithi062 ceph-mon[90553]: failed to encode map e114 with expected crc 2024-03-24T11:26:01.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:00 smithi139 ceph-mon[91894]: pgmap v77: 161 pgs: 161 active+clean; 585 KiB data, 114 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:26:01.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:00 smithi139 ceph-mon[91894]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T11:26:01.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:00 smithi139 ceph-mon[91894]: failed to encode map e114 with expected crc 2024-03-24T11:26:01.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:00 smithi139 ceph-mon[91894]: failed to encode map e114 with expected crc 2024-03-24T11:26:01.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:00 smithi139 ceph-mon[91894]: osdmap e114: 8 total, 7 up, 8 in 2024-03-24T11:26:01.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:00 smithi139 ceph-mon[91894]: failed to encode map e114 with expected crc 2024-03-24T11:26:01.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:00 smithi139 ceph-mon[91894]: failed to encode map e114 with expected crc 2024-03-24T11:26:01.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:00 smithi139 ceph-mon[91894]: failed to encode map e114 with expected crc 2024-03-24T11:26:01.404 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:26:01 smithi062 podman[113140]: 2024-03-24 11:26:01.046654032 +0000 UTC m=+0.610778210 container remove 61e78e75257c59caf926fe47afc07a9a349c3e5d30dfbeba6180bef63b5c0307 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-24T11:26:01.666 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:26:01 smithi062 podman[113228]: 2024-03-24 11:26:01.30874742 +0000 UTC m=+0.018803811 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:26:01.667 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:26:01 smithi062 podman[113228]: 2024-03-24 11:26:01.40599615 +0000 UTC m=+0.116052572 container create 819a41531fa6293e763160b615336dfbbe911a7be97d632392ec23d7e018650a (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0-deactivate, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-db0330b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-24T11:26:01.667 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:26:01 smithi062 podman[113228]: 2024-03-24 11:26:01.591738436 +0000 UTC m=+0.301794823 container init 819a41531fa6293e763160b615336dfbbe911a7be97d632392ec23d7e018650a (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-03-24T11:26:01.667 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:26:01 smithi062 podman[113228]: 2024-03-24 11:26:01.594897997 +0000 UTC m=+0.304954383 container start 819a41531fa6293e763160b615336dfbbe911a7be97d632392ec23d7e018650a (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-db0330b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T11:26:01.667 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:26:01 smithi062 podman[113228]: 2024-03-24 11:26:01.666696318 +0000 UTC m=+0.376752715 container attach 819a41531fa6293e763160b615336dfbbe911a7be97d632392ec23d7e018650a (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-24T11:26:01.977 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:26:01 smithi062 podman[113228]: 2024-03-24 11:26:01.903017256 +0000 UTC m=+0.613073635 container died 819a41531fa6293e763160b615336dfbbe911a7be97d632392ec23d7e018650a (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-24T11:26:02.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:01 smithi139 ceph-mon[91894]: failed to encode map e115 with expected crc 2024-03-24T11:26:02.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:01 smithi139 ceph-mon[91894]: osdmap e115: 8 total, 7 up, 8 in 2024-03-24T11:26:02.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:01 smithi139 ceph-mon[91894]: failed to encode map e115 with expected crc 2024-03-24T11:26:02.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:01 smithi139 ceph-mon[91894]: failed to encode map e115 with expected crc 2024-03-24T11:26:02.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:01 smithi139 ceph-mon[91894]: failed to encode map e115 with expected crc 2024-03-24T11:26:02.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:01 smithi139 ceph-mon[91894]: failed to encode map e115 with expected crc 2024-03-24T11:26:02.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:01 smithi139 ceph-mon[91894]: failed to encode map e115 with expected crc 2024-03-24T11:26:02.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:01 smithi139 ceph-mon[91894]: failed to encode map e115 with expected crc 2024-03-24T11:26:02.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:01 smithi139 ceph-mon[91894]: failed to encode map e115 with expected crc 2024-03-24T11:26:02.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:01 smithi139 ceph-mon[91894]: failed to encode map e115 with expected crc 2024-03-24T11:26:02.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:01 smithi062 ceph-mon[98643]: failed to encode map e115 with expected crc 2024-03-24T11:26:02.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:01 smithi062 ceph-mon[98643]: osdmap e115: 8 total, 7 up, 8 in 2024-03-24T11:26:02.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:01 smithi062 ceph-mon[98643]: failed to encode map e115 with expected crc 2024-03-24T11:26:02.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:01 smithi062 ceph-mon[98643]: failed to encode map e115 with expected crc 2024-03-24T11:26:02.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:01 smithi062 ceph-mon[98643]: failed to encode map e115 with expected crc 2024-03-24T11:26:02.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:01 smithi062 ceph-mon[98643]: failed to encode map e115 with expected crc 2024-03-24T11:26:02.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:01 smithi062 ceph-mon[98643]: failed to encode map e115 with expected crc 2024-03-24T11:26:02.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:01 smithi062 ceph-mon[98643]: failed to encode map e115 with expected crc 2024-03-24T11:26:02.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:01 smithi062 ceph-mon[98643]: failed to encode map e115 with expected crc 2024-03-24T11:26:02.410 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:01 smithi062 ceph-mon[98643]: failed to encode map e115 with expected crc 2024-03-24T11:26:02.410 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:01 smithi062 ceph-mon[90553]: failed to encode map e115 with expected crc 2024-03-24T11:26:02.410 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:01 smithi062 ceph-mon[90553]: osdmap e115: 8 total, 7 up, 8 in 2024-03-24T11:26:02.410 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:01 smithi062 ceph-mon[90553]: failed to encode map e115 with expected crc 2024-03-24T11:26:02.410 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:01 smithi062 ceph-mon[90553]: failed to encode map e115 with expected crc 2024-03-24T11:26:02.410 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:01 smithi062 ceph-mon[90553]: failed to encode map e115 with expected crc 2024-03-24T11:26:02.410 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:01 smithi062 ceph-mon[90553]: failed to encode map e115 with expected crc 2024-03-24T11:26:02.410 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:01 smithi062 ceph-mon[90553]: failed to encode map e115 with expected crc 2024-03-24T11:26:02.410 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:01 smithi062 ceph-mon[90553]: failed to encode map e115 with expected crc 2024-03-24T11:26:02.410 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:01 smithi062 ceph-mon[90553]: failed to encode map e115 with expected crc 2024-03-24T11:26:02.410 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:01 smithi062 ceph-mon[90553]: failed to encode map e115 with expected crc 2024-03-24T11:26:02.727 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:26:02 smithi062 podman[113254]: 2024-03-24 11:26:02.521594873 +0000 UTC m=+0.613527864 container remove 819a41531fa6293e763160b615336dfbbe911a7be97d632392ec23d7e018650a (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-24T11:26:02.727 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:26:02 smithi062 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.0.service: Deactivated successfully. 2024-03-24T11:26:02.727 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:26:02 smithi062 systemd[1]: Stopped Ceph osd.0 for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:26:02.727 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:26:02 smithi062 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.0.service: Consumed 1.787s CPU time. 2024-03-24T11:26:03.117 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:02 smithi062 ceph-mon[90553]: pgmap v80: 161 pgs: 21 stale+active+clean, 140 active+clean; 585 KiB data, 114 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-24T11:26:03.118 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:02 smithi062 ceph-mon[98643]: pgmap v80: 161 pgs: 21 stale+active+clean, 140 active+clean; 585 KiB data, 114 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-24T11:26:03.118 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:26:02 smithi062 systemd[1]: Starting Ceph osd.0 for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:26:03.118 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:26:02 smithi062 podman[113354]: 2024-03-24 11:26:02.874558493 +0000 UTC m=+0.017857562 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:26:03.118 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:26:02 smithi062 podman[113354]: 2024-03-24 11:26:02.989386718 +0000 UTC m=+0.132685823 container create b8145c7e83286d2e2a2b7a2acedd1de8e4b5be322f7238abc7d30b09c08de6cf (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0-activate, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-24T11:26:03.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:02 smithi139 ceph-mon[91894]: pgmap v80: 161 pgs: 21 stale+active+clean, 140 active+clean; 585 KiB data, 114 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-24T11:26:03.477 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:26:03 smithi062 podman[113354]: 2024-03-24 11:26:03.225582487 +0000 UTC m=+0.368881569 container init b8145c7e83286d2e2a2b7a2acedd1de8e4b5be322f7238abc7d30b09c08de6cf (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0-activate, CEPH_POINT_RELEASE=, RELEASE=squid-db0330b, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-24T11:26:03.477 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:26:03 smithi062 podman[113354]: 2024-03-24 11:26:03.228789847 +0000 UTC m=+0.372088922 container start b8145c7e83286d2e2a2b7a2acedd1de8e4b5be322f7238abc7d30b09c08de6cf (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0-activate, RELEASE=squid-db0330b, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-24T11:26:03.477 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:26:03 smithi062 podman[113354]: 2024-03-24 11:26:03.308846877 +0000 UTC m=+0.452145955 container attach b8145c7e83286d2e2a2b7a2acedd1de8e4b5be322f7238abc7d30b09c08de6cf (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0-activate, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T11:26:03.977 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:26:03 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0-activate[113368]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-24T11:26:03.977 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:26:03 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0-activate[113368]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-24T11:26:03.977 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:26:03 smithi062 bash[113354]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-24T11:26:03.977 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:26:03 smithi062 bash[113354]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-24T11:26:03.977 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:26:03 smithi062 bash[113354]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-24T11:26:03.977 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:26:03 smithi062 bash[113354]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-24T11:26:03.977 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:26:03 smithi062 bash[113354]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-03-24T11:26:03.977 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:26:03 smithi062 bash[113354]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-24T11:26:03.978 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:26:03 smithi062 bash[113354]: --> ceph-volume raw activate successful for osd ID: 0 2024-03-24T11:26:03.978 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:26:03 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0-activate[113368]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-24T11:26:03.978 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:26:03 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0-activate[113368]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-24T11:26:03.978 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:26:03 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0-activate[113368]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-03-24T11:26:03.978 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:26:03 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0-activate[113368]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-24T11:26:03.978 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:26:03 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0-activate[113368]: --> ceph-volume raw activate successful for osd ID: 0 2024-03-24T11:26:03.978 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:26:03 smithi062 podman[113354]: 2024-03-24 11:26:03.699213366 +0000 UTC m=+0.842512431 container died b8145c7e83286d2e2a2b7a2acedd1de8e4b5be322f7238abc7d30b09c08de6cf (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0-activate, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=) 2024-03-24T11:26:04.291 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:03 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:26:04.292 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:03 smithi062 ceph-mon[98643]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:26:04.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:03 smithi139 ceph-mon[91894]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:26:04.633 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:26:04 smithi062 podman[113500]: 2024-03-24 11:26:04.291798346 +0000 UTC m=+0.586113765 container remove b8145c7e83286d2e2a2b7a2acedd1de8e4b5be322f7238abc7d30b09c08de6cf (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T11:26:04.633 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:26:04 smithi062 podman[113565]: 2024-03-24 11:26:04.405633087 +0000 UTC m=+0.019102930 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:26:04.634 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:26:04 smithi062 podman[113565]: 2024-03-24 11:26:04.517840441 +0000 UTC m=+0.131310270 container create dfe4920b49c4d7a791c09566e939cd2e31f60c9788ae7ac0f7c0bc049d2fb1a1 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0, io.buildah.version=1.33.5, RELEASE=squid-db0330b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-24T11:26:04.977 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:26:04 smithi062 podman[113565]: 2024-03-24 11:26:04.720386609 +0000 UTC m=+0.333856451 container init dfe4920b49c4d7a791c09566e939cd2e31f60c9788ae7ac0f7c0bc049d2fb1a1 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-db0330b, GIT_CLEAN=True) 2024-03-24T11:26:04.977 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:26:04 smithi062 podman[113565]: 2024-03-24 11:26:04.724437378 +0000 UTC m=+0.337907207 container start dfe4920b49c4d7a791c09566e939cd2e31f60c9788ae7ac0f7c0bc049d2fb1a1 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0, org.label-schema.build-date=20240307, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-db0330b, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T11:26:04.977 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:26:04 smithi062 ceph-osd[113584]: -- 172.21.15.62:0/1361505468 <== mon.1 v2:172.21.15.62:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558a3d1771e0 con 0x558a3d149000 2024-03-24T11:26:04.977 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:26:04 smithi062 bash[113565]: dfe4920b49c4d7a791c09566e939cd2e31f60c9788ae7ac0f7c0bc049d2fb1a1 2024-03-24T11:26:04.977 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:26:04 smithi062 systemd[1]: Started Ceph osd.0 for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:26:05.243 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:04 smithi062 ceph-mon[90553]: pgmap v81: 161 pgs: 20 active+undersized+degraded, 38 active+undersized, 103 active+clean; 585 KiB data, 115 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 68/630 objects degraded (10.794%) 2024-03-24T11:26:05.243 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:04 smithi062 ceph-mon[90553]: Health check failed: Degraded data redundancy: 68/630 objects degraded (10.794%), 20 pgs degraded (PG_DEGRADED) 2024-03-24T11:26:05.243 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:05.243 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:05.243 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:05.243 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:05.243 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:05.244 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:04 smithi062 ceph-mon[98643]: pgmap v81: 161 pgs: 20 active+undersized+degraded, 38 active+undersized, 103 active+clean; 585 KiB data, 115 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 68/630 objects degraded (10.794%) 2024-03-24T11:26:05.244 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:04 smithi062 ceph-mon[98643]: Health check failed: Degraded data redundancy: 68/630 objects degraded (10.794%), 20 pgs degraded (PG_DEGRADED) 2024-03-24T11:26:05.244 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:05.244 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:05.244 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:05.244 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:05.244 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:05.259 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:04 smithi139 ceph-mon[91894]: pgmap v81: 161 pgs: 20 active+undersized+degraded, 38 active+undersized, 103 active+clean; 585 KiB data, 115 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 68/630 objects degraded (10.794%) 2024-03-24T11:26:05.259 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:04 smithi139 ceph-mon[91894]: Health check failed: Degraded data redundancy: 68/630 objects degraded (10.794%), 20 pgs degraded (PG_DEGRADED) 2024-03-24T11:26:05.259 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:05.259 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:05.259 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:05.259 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:05.259 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:05.727 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0[113580]: 2024-03-24T11:26:05.262+0000 7f0a637507c0 -1 Falling back to public interface 2024-03-24T11:26:06.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:06.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:06.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:06.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:06.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:06.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:06.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:06.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:06.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:06.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:06.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:06.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:06.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:06.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:06.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:06.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:06.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:06.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:06.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:06.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:06.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:06.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:06.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:06.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:06.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:06.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:06.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:06.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:06.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:06.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:06.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:06.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:06.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:06.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:06.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:06.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:06.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:06.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:06.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:06.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:06.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:06.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:06.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:06.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:06.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:06.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:06.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:06.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:06.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:06.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:06.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:06.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:06.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:06.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:06.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:06.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:06.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:06.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:06.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:06.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:06.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:06.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:06.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:06.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:06.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:06.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:06.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:06.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:06.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:06.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:06.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:06.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:06.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:06.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:06.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:06.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:06.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:06.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:06.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:06.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:06.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:06.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:06.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:06.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:06.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:05 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:06.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:06.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:06.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:06.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:06.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:06.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:05 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:06.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:06.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:06.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:06.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:06.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:06.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:06.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:06.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:06.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:06.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:06.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:06.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:06.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:06.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:06.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:06.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:06.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:06.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:06.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:06.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:06.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:06.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:06.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:06.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:06.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:06.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:06.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:06.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:06.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:06.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:06.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:06.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:06.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:06.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:06.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:07.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: pgmap v82: 161 pgs: 20 active+undersized+degraded, 38 active+undersized, 103 active+clean; 585 KiB data, 115 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 68/630 objects degraded (10.794%) 2024-03-24T11:26:07.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:07.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:07.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:07.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:07.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:07.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:07.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:07.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:07.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:07.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:07.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:07.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:07.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:07.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:07.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:07.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:07.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:07.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:07.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:07.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:07.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:07.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:07.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:07.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:07.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:07.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:07.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:07.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:07.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:07.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:07.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:07.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:07.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:07.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:07.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:07.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:07.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:07.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:07.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:07.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:07.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:07.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:07.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:07.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:07.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:07.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:07.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:07.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:07.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:07.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:07.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:07.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:07.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:07.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:07.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:07.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:07.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:07.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: pgmap v82: 161 pgs: 20 active+undersized+degraded, 38 active+undersized, 103 active+clean; 585 KiB data, 115 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 68/630 objects degraded (10.794%) 2024-03-24T11:26:07.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:07.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:07.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:07.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:07.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:07.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:07.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:07.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:07.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:07.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:07.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:07.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:07.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:07.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:07.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:07.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:07.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:07.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:07.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:07.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:07.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:07.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:07.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:07.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:07.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:07.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:07.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:07.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:07.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:07.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:07.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:07.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:07.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:07.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:07.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:07.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:07.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:07.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:07.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:07.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:07.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:07.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:07.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:07.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:07.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:07.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:07.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:07.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:07.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:07.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:07.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:07.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:07.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:07.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:07.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:07.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:07.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:07.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: pgmap v82: 161 pgs: 20 active+undersized+degraded, 38 active+undersized, 103 active+clean; 585 KiB data, 115 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 68/630 objects degraded (10.794%) 2024-03-24T11:26:07.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:07.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:07.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:07.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:07.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:07.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:07.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:07.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:07.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:07.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:07.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:07.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:07.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:07.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:07.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:07.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:07.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:07.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:07.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:07.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:07.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:07.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:07.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:07.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:07.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:07.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:07.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:07.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:07.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:07.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:07.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:07.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:07.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:07.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:07.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:07.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:07.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:07.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:07.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:07.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:07.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:07.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:07.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:07.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:07.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:07.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:07.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:07.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:07.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:07.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:07.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:07.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:07.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:07.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:07.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:07.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:07.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:08.186 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:26:07 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0[113580]: 2024-03-24T11:26:07.991+0000 7f0a637507c0 -1 osd.0 113 log_to_monitors true 2024-03-24T11:26:08.186 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:08.187 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:08.187 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:08.187 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:08.187 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:08.187 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:08.187 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:08.187 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:08.187 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:08.187 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:08.187 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:08.187 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:08.187 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:08.187 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:08.187 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:08.188 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:08.188 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:08.188 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:08.188 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:08.188 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:08.188 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:08.188 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:08.188 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:08.188 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:08.188 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:08.188 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:08.188 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:08.188 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:08.188 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:08.188 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:08.189 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:08.189 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:08.189 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:08.189 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:08.189 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:08.189 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:08.189 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:08.189 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:08.189 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:08.189 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:08.189 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:08.190 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:08.190 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:08.190 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:08.190 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:08.190 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:08.190 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:08.190 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:08.190 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:08.190 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:08.190 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:08.190 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:08.190 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:08.190 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:08.190 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:08.190 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:08.191 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:08.191 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:08.191 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:08.191 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:08.191 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:08.191 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:08.191 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:08.191 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:08.191 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:08.191 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:08.191 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:08.192 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:08.192 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:08.192 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:08.192 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:08.192 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:08.192 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:08.192 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:08.192 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:08.192 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:08.192 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:08.192 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:08.192 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:08.192 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:08.192 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:08.193 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:08.193 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:08.193 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:08.193 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:08.193 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:08.193 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:08.193 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:08.193 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:08.193 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:08.193 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:08.194 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:08.194 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:08.194 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:08.194 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:08.194 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:08.194 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:08.194 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:08.194 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:08.194 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:08.194 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:08.194 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:08.194 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:08.194 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:08.194 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:08.195 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:08.195 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:08.195 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:08.195 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:08.195 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:08.195 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:08.195 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:08.195 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:08.195 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:08.195 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:08.195 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:08.195 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:08.195 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:08.195 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:08.195 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:08.196 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:08.196 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:08.196 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:08.196 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='osd.0 [v2:172.21.15.62:6802/3283936818,v1:172.21.15.62:6803/3283936818]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-24T11:26:08.196 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[98643]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-24T11:26:08.196 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:08.196 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:08.196 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:08.196 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:08.196 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:08.196 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:08.196 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:08.197 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='osd.0 [v2:172.21.15.62:6802/3283936818,v1:172.21.15.62:6803/3283936818]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-24T11:26:08.197 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:08 smithi062 ceph-mon[90553]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-24T11:26:08.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:08.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:08.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:08.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:08.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:08.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:08.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:08.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:08.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:08.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:08.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:08.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:08.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:08.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:08.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:08.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:08.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:08.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:08.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:08.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:08.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:08.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:08.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:08.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:08.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:08.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:08.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:08.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:08.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:08.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:08.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:08.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:08.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:08.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:08.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:08.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:08.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:08.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:08.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:08.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:08.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:08.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:08.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:08.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:08.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:08.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:08.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:08.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:08.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:08.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:08.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:08.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:08.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:08.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:08.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:08.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:08.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:08.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:08.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:08.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:08.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:08.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:08.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:08.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:08.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:08.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='osd.0 [v2:172.21.15.62:6802/3283936818,v1:172.21.15.62:6803/3283936818]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-24T11:26:08.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:08 smithi139 ceph-mon[91894]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-24T11:26:09.465 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:09.465 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: pgmap v83: 161 pgs: 20 active+undersized+degraded, 38 active+undersized, 103 active+clean; 585 KiB data, 115 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 68/630 objects degraded (10.794%) 2024-03-24T11:26:09.465 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:09.465 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:09.465 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:09.465 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:09.465 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:09.466 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:09.466 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:09.466 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:09.466 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:09.466 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:09.466 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:09.466 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:09.466 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:09.466 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:09.466 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:09.466 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:09.466 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:09.466 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:09.466 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:09.466 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:09.467 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:09.467 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:09.467 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:09.467 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:09.467 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:09.467 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:09.467 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:09.467 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:09.467 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:09.467 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: pgmap v83: 161 pgs: 20 active+undersized+degraded, 38 active+undersized, 103 active+clean; 585 KiB data, 115 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 68/630 objects degraded (10.794%) 2024-03-24T11:26:09.467 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:09.468 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:09.468 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:09.468 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:09.468 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:09.468 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:09.468 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:09.468 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:09.468 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:09.468 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:09.468 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:09.468 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:09.468 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:09.468 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:09.468 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:09.469 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:09.469 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:09.469 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:09.469 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:09.469 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:09.469 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:09.469 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:09.469 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:09.469 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:09.469 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:09.469 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:09.469 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:09.469 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:09.470 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:09.470 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:09.470 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:09.470 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:09.470 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:09.470 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:09.470 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:09.470 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:09.470 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:09.470 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:09.470 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:09.470 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:09.470 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:09.471 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:09.471 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:09.471 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:09.471 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:09.471 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:09.471 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:09.471 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:09.471 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:09.471 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:09.471 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-24T11:26:09.471 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: osdmap e116: 8 total, 7 up, 8 in 2024-03-24T11:26:09.471 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: from='osd.0 [v2:172.21.15.62:6802/3283936818,v1:172.21.15.62:6803/3283936818]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:26:09.471 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:26:09.471 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:09.472 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:09.472 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:09.472 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:09.472 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:09.472 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:09.472 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:09.472 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:09.472 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:09.472 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:09.472 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:09.472 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:09.472 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:09.472 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:09.472 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:09.473 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:09.473 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:09.473 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:09.473 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:09.473 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:09.473 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:09.473 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:09.473 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:09.473 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-24T11:26:09.473 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: osdmap e116: 8 total, 7 up, 8 in 2024-03-24T11:26:09.473 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: from='osd.0 [v2:172.21.15.62:6802/3283936818,v1:172.21.15.62:6803/3283936818]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:26:09.473 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:26:09.473 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:09.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:09.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: pgmap v83: 161 pgs: 20 active+undersized+degraded, 38 active+undersized, 103 active+clean; 585 KiB data, 115 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 68/630 objects degraded (10.794%) 2024-03-24T11:26:09.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:09.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:09.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:09.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:09.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:09.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:09.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:09.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:09.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:09.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:09.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:09.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:09.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:09.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:09.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:09.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:09.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:09.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:09.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:09.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:09.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:09.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:09.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:09.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:09.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:09.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:09.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:09.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:09.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:09.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:09.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:09.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:09.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:09.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:09.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:09.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:09.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:09.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:09.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:09.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:09.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:09.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:09.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:09.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:09.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:09.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:09.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:09.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:09.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:09.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:09.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-24T11:26:09.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: osdmap e116: 8 total, 7 up, 8 in 2024-03-24T11:26:09.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: from='osd.0 [v2:172.21.15.62:6802/3283936818,v1:172.21.15.62:6803/3283936818]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:26:09.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:26:09.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:09.726 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:26:09 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0[113580]: 2024-03-24T11:26:09.463+0000 7f0a50ba0700 -1 osd.0 113 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T11:26:10.405 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:10.405 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:10.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:10.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:10.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:10.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:10.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:10.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:10.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: failed to encode map e116 with expected crc 2024-03-24T11:26:10.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: failed to encode map e116 with expected crc 2024-03-24T11:26:10.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: failed to encode map e116 with expected crc 2024-03-24T11:26:10.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:10.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:10.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:10.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: failed to encode map e116 with expected crc 2024-03-24T11:26:10.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: pgmap v85: 161 pgs: 20 active+undersized+degraded, 38 active+undersized, 103 active+clean; 585 KiB data, 115 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 68/630 objects degraded (10.794%) 2024-03-24T11:26:10.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: failed to encode map e116 with expected crc 2024-03-24T11:26:10.407 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: failed to encode map e116 with expected crc 2024-03-24T11:26:10.407 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: failed to encode map e116 with expected crc 2024-03-24T11:26:10.407 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:10.407 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:10.407 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: from='osd.0 ' entity='osd.0' 2024-03-24T11:26:10.407 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:10.407 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:10.407 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:10.407 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:10.407 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:10.407 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:10.407 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:10.407 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:10.407 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:10.407 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:10.408 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:10.408 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:10.408 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:10.408 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:10.408 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:10.408 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:10.408 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:10.408 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:10.408 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:10.408 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: failed to encode map e116 with expected crc 2024-03-24T11:26:10.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: failed to encode map e116 with expected crc 2024-03-24T11:26:10.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: failed to encode map e116 with expected crc 2024-03-24T11:26:10.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:10.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:10.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:10.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: failed to encode map e116 with expected crc 2024-03-24T11:26:10.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: pgmap v85: 161 pgs: 20 active+undersized+degraded, 38 active+undersized, 103 active+clean; 585 KiB data, 115 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 68/630 objects degraded (10.794%) 2024-03-24T11:26:10.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: failed to encode map e116 with expected crc 2024-03-24T11:26:10.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: failed to encode map e116 with expected crc 2024-03-24T11:26:10.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: failed to encode map e116 with expected crc 2024-03-24T11:26:10.410 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:10.410 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:10.410 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: from='osd.0 ' entity='osd.0' 2024-03-24T11:26:10.410 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:10.410 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:10.410 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:10.410 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:10.410 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:10.410 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:10.410 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:10.410 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:10.410 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:10.410 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:10.410 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:10.411 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:10.411 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:10.411 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:10.411 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:10.411 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:10.411 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:10.411 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:10.411 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:10.411 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:10.411 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:10.411 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:10.411 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:10.412 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:10.412 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:10.412 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:10.412 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:10.412 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:10.412 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:10.412 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:10.412 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:10.412 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:10.412 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:10.412 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:10.413 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:10.413 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:10.413 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:10.413 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:10.413 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:10.413 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:10.414 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:10.414 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:10.414 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:10.414 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:10.414 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:10.414 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:10.414 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:10.414 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:10.414 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:10.414 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:10.414 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:10.414 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:10.415 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:10.415 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:10.415 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:10.415 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:10.415 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:10.415 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:10.415 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:10.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:10.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:10.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:10.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:10.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:10.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:10.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:10.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:10.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: failed to encode map e116 with expected crc 2024-03-24T11:26:10.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: failed to encode map e116 with expected crc 2024-03-24T11:26:10.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: failed to encode map e116 with expected crc 2024-03-24T11:26:10.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:10.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:10.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:10.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: failed to encode map e116 with expected crc 2024-03-24T11:26:10.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: pgmap v85: 161 pgs: 20 active+undersized+degraded, 38 active+undersized, 103 active+clean; 585 KiB data, 115 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 68/630 objects degraded (10.794%) 2024-03-24T11:26:10.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: failed to encode map e116 with expected crc 2024-03-24T11:26:10.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: failed to encode map e116 with expected crc 2024-03-24T11:26:10.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: failed to encode map e116 with expected crc 2024-03-24T11:26:10.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:10.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:10.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: from='osd.0 ' entity='osd.0' 2024-03-24T11:26:10.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:10.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:10.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:10.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:10.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:10.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:10.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:10.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:10.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:10.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:10.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:10.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:10.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:10.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:10.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:10.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:10.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:10.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:10.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:10.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:10.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:10.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:10.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:10.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:10.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:10.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:10.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:10.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:10.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:10.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:10.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:10.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:10.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:10.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:10.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:11.122 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:26:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:26:10] "GET /metrics HTTP/1.1" 200 38495 "" "Prometheus/2.33.4" 2024-03-24T11:26:11.428 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:11.428 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:11.428 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:11.428 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:11.428 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:11.428 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:11.428 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:11.428 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:11.428 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:11.428 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:11.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:11.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:11.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:11.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:11.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:11.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:11.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:11.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:11.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:11.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T11:26:11.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: failed to encode map e117 with expected crc 2024-03-24T11:26:11.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: failed to encode map e117 with expected crc 2024-03-24T11:26:11.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: osd.0 [v2:172.21.15.62:6802/3283936818,v1:172.21.15.62:6803/3283936818] boot 2024-03-24T11:26:11.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: failed to encode map e117 with expected crc 2024-03-24T11:26:11.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: osdmap e117: 8 total, 8 up, 8 in 2024-03-24T11:26:11.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T11:26:11.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:11.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:11.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:11.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:11.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:11.430 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:11.430 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:11.430 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:11.431 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:11.431 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:11.431 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:11.431 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:11.431 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:11.431 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:11.431 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:11.431 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:11.431 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:11.431 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:11.431 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:11.431 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:11.431 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:11.431 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:11.431 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:11.432 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:11.432 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T11:26:11.432 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: failed to encode map e117 with expected crc 2024-03-24T11:26:11.432 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: failed to encode map e117 with expected crc 2024-03-24T11:26:11.432 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: osd.0 [v2:172.21.15.62:6802/3283936818,v1:172.21.15.62:6803/3283936818] boot 2024-03-24T11:26:11.432 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: failed to encode map e117 with expected crc 2024-03-24T11:26:11.432 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: osdmap e117: 8 total, 8 up, 8 in 2024-03-24T11:26:11.432 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T11:26:11.432 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:11.432 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:11.432 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:11.432 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:11.432 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:11.433 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:11.433 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:11.433 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:11.433 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:11.433 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:11.433 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:11.433 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:11.433 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:11.433 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:11.433 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:11.433 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:11.434 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:11.434 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:11.434 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:11.434 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:11.434 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:11.434 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:11.434 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:11.434 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:11.434 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:11.434 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:11.434 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:11.434 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:11.434 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:11.435 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:11.435 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:11.435 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:11.435 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:11.435 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:11.435 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:11.435 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:11.435 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:11.435 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:11.435 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:11.435 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:11.435 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:11.436 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:11.436 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:11.436 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:11.436 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:11.436 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:11.436 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:11.436 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:11.436 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:11.436 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:11.436 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:11.436 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:11.436 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:11.436 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:11.436 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:11.436 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:11.437 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:11.437 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:11.437 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:11.437 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:11.437 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:11.437 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:11.437 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:11.437 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:11.437 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:11.437 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:11.437 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:11.437 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:11.437 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:11.437 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:11.437 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:11.437 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:11.438 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:11.438 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:11.438 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:11.438 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:11.438 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:11.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:11.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:11.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:11.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:11.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:11.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:11.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:11.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:11.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:11.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:11.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:11.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:11.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:11.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:11.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:11.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:11.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:11.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:11.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:11.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T11:26:11.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: failed to encode map e117 with expected crc 2024-03-24T11:26:11.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: failed to encode map e117 with expected crc 2024-03-24T11:26:11.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: osd.0 [v2:172.21.15.62:6802/3283936818,v1:172.21.15.62:6803/3283936818] boot 2024-03-24T11:26:11.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: failed to encode map e117 with expected crc 2024-03-24T11:26:11.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: osdmap e117: 8 total, 8 up, 8 in 2024-03-24T11:26:11.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T11:26:11.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:11.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:11.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:11.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:11.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:11.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:11.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:11.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:11.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:11.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:11.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:11.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:11.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:11.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:11.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:11.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:11.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:11.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:11.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:11.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:11.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:11.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:11.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:11.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:11.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:11.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:11.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:11.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:11.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:11.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:11.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:11.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:11.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:11.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:11.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:11.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:11.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:11.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:11.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:11.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:11.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:12.402 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: failed to encode map e117 with expected crc 2024-03-24T11:26:12.402 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: failed to encode map e117 with expected crc 2024-03-24T11:26:12.402 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:12.402 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: pgmap v87: 161 pgs: 20 active+undersized+degraded, 38 active+undersized, 103 active+clean; 585 KiB data, 115 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 68/630 objects degraded (10.794%) 2024-03-24T11:26:12.402 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:12.402 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:12.403 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:12.403 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:26:12.403 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:12.403 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:12.403 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:12.403 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:12.403 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:12.403 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:12.403 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:12.403 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:12.403 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:12.403 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:12.403 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:12.403 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:12.403 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:12.404 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:12.404 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:12.404 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:12.404 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: failed to encode map e118 with expected crc 2024-03-24T11:26:12.404 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: failed to encode map e118 with expected crc 2024-03-24T11:26:12.404 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: osdmap e118: 8 total, 8 up, 8 in 2024-03-24T11:26:12.404 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: failed to encode map e118 with expected crc 2024-03-24T11:26:12.404 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: failed to encode map e118 with expected crc 2024-03-24T11:26:12.404 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: failed to encode map e118 with expected crc 2024-03-24T11:26:12.404 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: failed to encode map e118 with expected crc 2024-03-24T11:26:12.404 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: failed to encode map e118 with expected crc 2024-03-24T11:26:12.404 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: failed to encode map e118 with expected crc 2024-03-24T11:26:12.405 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:12.405 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:12.405 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:12.405 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:12.405 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:12.405 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:12.405 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:12.405 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:12.405 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:12.405 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:12.405 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:12.405 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:12.405 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:12.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:12.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:12.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:12.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:12.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:12.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:12.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:12.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:12.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:12.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:12.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:12.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:12.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:12.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:12.407 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:12.407 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:12.407 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:12.407 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:12.407 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:12.407 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:12.407 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:12.407 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:12.407 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: failed to encode map e117 with expected crc 2024-03-24T11:26:12.407 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: failed to encode map e117 with expected crc 2024-03-24T11:26:12.408 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:12.408 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: pgmap v87: 161 pgs: 20 active+undersized+degraded, 38 active+undersized, 103 active+clean; 585 KiB data, 115 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 68/630 objects degraded (10.794%) 2024-03-24T11:26:12.408 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:12.408 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:12.408 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:12.408 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:26:12.408 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:12.408 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:12.408 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:12.408 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:12.408 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:12.408 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:12.408 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:12.408 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:12.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:12.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:12.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:12.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:12.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:12.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:12.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:12.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:12.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: failed to encode map e118 with expected crc 2024-03-24T11:26:12.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: failed to encode map e118 with expected crc 2024-03-24T11:26:12.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: osdmap e118: 8 total, 8 up, 8 in 2024-03-24T11:26:12.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: failed to encode map e118 with expected crc 2024-03-24T11:26:12.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: failed to encode map e118 with expected crc 2024-03-24T11:26:12.410 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: failed to encode map e118 with expected crc 2024-03-24T11:26:12.410 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: failed to encode map e118 with expected crc 2024-03-24T11:26:12.410 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: failed to encode map e118 with expected crc 2024-03-24T11:26:12.410 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: failed to encode map e118 with expected crc 2024-03-24T11:26:12.410 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:12.410 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:12.410 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:12.410 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:12.410 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:12.410 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:12.410 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:12.410 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:12.410 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:12.410 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:12.411 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:12.411 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:12.411 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:12.411 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:12.411 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:12.411 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:12.411 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:12.411 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:12.411 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:12.411 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:12.411 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:12.411 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:12.411 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:12.411 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:12.411 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:12.412 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:12.412 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:12.412 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:12.412 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:12.412 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:12.412 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:12.412 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:12.412 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:12.412 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:12.412 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:12.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: failed to encode map e117 with expected crc 2024-03-24T11:26:12.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: failed to encode map e117 with expected crc 2024-03-24T11:26:12.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:12.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: pgmap v87: 161 pgs: 20 active+undersized+degraded, 38 active+undersized, 103 active+clean; 585 KiB data, 115 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 68/630 objects degraded (10.794%) 2024-03-24T11:26:12.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:12.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:12.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:12.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:26:12.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:12.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:12.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:12.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:12.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:12.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:12.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:12.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:12.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:12.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:12.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:12.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:12.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:12.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:12.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:12.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:12.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: failed to encode map e118 with expected crc 2024-03-24T11:26:12.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: failed to encode map e118 with expected crc 2024-03-24T11:26:12.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: osdmap e118: 8 total, 8 up, 8 in 2024-03-24T11:26:12.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: failed to encode map e118 with expected crc 2024-03-24T11:26:12.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: failed to encode map e118 with expected crc 2024-03-24T11:26:12.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: failed to encode map e118 with expected crc 2024-03-24T11:26:12.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: failed to encode map e118 with expected crc 2024-03-24T11:26:12.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: failed to encode map e118 with expected crc 2024-03-24T11:26:12.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: failed to encode map e118 with expected crc 2024-03-24T11:26:12.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:12.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:12.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:12.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:12.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:12.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:12.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:12.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:12.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:12.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:12.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:12.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:12.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:12.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:12.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:12.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:12.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:12.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:12.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:12.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:12.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:12.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:12.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:12.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:12.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:12.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:12.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:12.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:12.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:12.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:12.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:12.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:12.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:12.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:12.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:13.284 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: failed to encode map e118 with expected crc 2024-03-24T11:26:13.284 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: failed to encode map e118 with expected crc 2024-03-24T11:26:13.284 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:13.284 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:13.284 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:13.284 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:13.285 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:13.285 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:13.285 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:13.285 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:26:13.285 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:13.285 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:13.285 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:13.285 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:13.285 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:13.285 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:13.285 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:13.286 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:13.286 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:13.286 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:13.286 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:13.286 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:13.286 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:13.286 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:13.286 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:13.286 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:13.286 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:13.286 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:13.287 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:13.287 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:13.287 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:13.287 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:13.287 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:13.287 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:13.287 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:13.287 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:13.287 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:13.287 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:13.287 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:13.288 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:13.288 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:13.288 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:13.288 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:13.288 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:13.288 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:13.288 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:13.288 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:13.288 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:13.288 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:13.288 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:13.289 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:13.289 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:13.289 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:13.289 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:13.289 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:13.289 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:13.289 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:13.289 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:13.289 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:13.289 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:13.289 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:13.289 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:13.289 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:13.289 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:13.289 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:13.290 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:13.290 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:13.290 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:13.290 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:13.383 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: failed to encode map e118 with expected crc 2024-03-24T11:26:13.383 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: failed to encode map e118 with expected crc 2024-03-24T11:26:13.383 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:13.384 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:13.384 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:13.384 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:13.384 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:13.384 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:13.384 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:13.384 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:26:13.384 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:13.384 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:13.384 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:13.384 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:13.384 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:13.384 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:13.384 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:13.384 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:13.384 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:13.385 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:13.385 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:13.385 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:13.385 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:13.385 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:13.385 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:13.385 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:13.385 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:13.385 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:13.385 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:13.385 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:13.386 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: failed to encode map e118 with expected crc 2024-03-24T11:26:13.386 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: failed to encode map e118 with expected crc 2024-03-24T11:26:13.386 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:13.386 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:13.386 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:13.386 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:13.386 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:13.386 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:13.386 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:13.386 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:26:13.386 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:13.386 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:13.386 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:13.387 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:13.387 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:13.387 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:13.387 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:13.387 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:13.387 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:13.387 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:13.387 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:13.387 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:13.387 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:13.387 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:13.387 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:13.387 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:13.387 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:13.387 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:13.388 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:13.388 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:13.388 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:13.388 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:13.388 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:13.388 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:13.388 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:13.388 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:13.388 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:13.388 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:13.388 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:13.389 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:13.389 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:13.389 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:13.389 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:13.389 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:13.389 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:13.389 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:13.389 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:13.389 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:13.389 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:13.390 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:13.390 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:13.390 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:13.390 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:13.390 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:13.390 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:13.390 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:13.390 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:13.390 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:13.390 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:13.390 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:13.390 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:13.391 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:13.391 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:13.391 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:13.391 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:13.391 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:13.391 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:13.391 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:13.391 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:13.391 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:13.391 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:13.391 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:13.391 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:13.391 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:13.391 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:13.391 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:13.392 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:13.392 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:13.392 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:13.392 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:13.392 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:13.392 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:13.392 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:13.392 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:13.392 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:13.392 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:13.392 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:13.393 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:13.393 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:13.393 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:13.393 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:13.393 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:13.393 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:13.393 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:13.393 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:13.393 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:13.393 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:13.394 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:13.394 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:13.394 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:13.394 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:13.394 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:13.394 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:13.394 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:13.394 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:13.394 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:13.394 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:13.394 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:14.219 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:14.220 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[90553]: pgmap v89: 161 pgs: 7 active+undersized+degraded, 14 active+undersized, 140 active+clean; 585 KiB data, 116 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 28/630 objects degraded (4.444%) 2024-03-24T11:26:14.220 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:14.220 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:14.220 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:14.220 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:14.220 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:14.220 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:14.220 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:14.220 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:14.220 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:14.220 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:14.220 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:14.220 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:14.221 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:14.221 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:14.221 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:14.221 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:14.221 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:14.221 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:14.221 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:14.221 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:14.221 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:14.221 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:14.221 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:14.221 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:14.221 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:14.221 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:14.222 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:14.222 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[98643]: pgmap v89: 161 pgs: 7 active+undersized+degraded, 14 active+undersized, 140 active+clean; 585 KiB data, 116 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 28/630 objects degraded (4.444%) 2024-03-24T11:26:14.222 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:14.222 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:14.222 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:14.222 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:14.223 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:14.223 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:14.223 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:14.223 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:14.223 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:14.223 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:14.223 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:14.223 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:14.223 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:14.223 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:14.223 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:14.223 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:14.223 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:14.223 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:14.224 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:14.224 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:14.224 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:14.224 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:14.224 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:14.224 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:14.224 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:14.224 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:14.224 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:14.224 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:14.224 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:14.224 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:14.224 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:14.224 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:14.224 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:14.224 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:14.225 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:14.225 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:14.225 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:14.225 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:14.225 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:14.225 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:14.225 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:14.225 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:14.225 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:14.225 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:14.225 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:14.225 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:14.225 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:14.225 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[98643]: Health check update: Degraded data redundancy: 28/630 objects degraded (4.444%), 7 pgs degraded (PG_DEGRADED) 2024-03-24T11:26:14.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:14.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:14.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:14.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:14.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:14.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:14.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:14.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:14.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:14.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:14.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:14.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:14.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:14.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:14.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:14.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:14.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:14.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:14.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:14.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:14.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:14.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:14 smithi062 ceph-mon[90553]: Health check update: Degraded data redundancy: 28/630 objects degraded (4.444%), 7 pgs degraded (PG_DEGRADED) 2024-03-24T11:26:14.301 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:14 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:14.302 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:14 smithi139 ceph-mon[91894]: pgmap v89: 161 pgs: 7 active+undersized+degraded, 14 active+undersized, 140 active+clean; 585 KiB data, 116 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 28/630 objects degraded (4.444%) 2024-03-24T11:26:14.302 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:14 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:14.302 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:14 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:14.302 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:14 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:14.302 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:14 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:14.302 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:14 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:14.302 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:14 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:14.302 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:14 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:14.302 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:14 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:14.302 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:14 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:14.302 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:14 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:14.302 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:14 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:14.302 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:14 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:14.303 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:14 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:14.303 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:14 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:14.303 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:14 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:14.303 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:14 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:14.303 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:14 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:14.303 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:14 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:14.303 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:14 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:14.303 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:14 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:14.303 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:14 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:14.303 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:14 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:14.303 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:14 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:14.303 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:14 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:14.303 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:14 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:14.303 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:14 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:14.304 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:14 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:14.304 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:14 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:14.304 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:14 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:14.304 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:14 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:14.304 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:14 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:14.304 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:14 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:14.304 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:14 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:14.304 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:14 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:14.304 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:14 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:14.304 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:14 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:14.304 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:14 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:14.304 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:14 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:14.304 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:14 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:14.304 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:14 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:14.304 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:14 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:14.305 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:14 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:14.305 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:14 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:14.305 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:14 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:14.305 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:14 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:14.305 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:14 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:14.305 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:14 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:14.305 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:14 smithi139 ceph-mon[91894]: Health check update: Degraded data redundancy: 28/630 objects degraded (4.444%), 7 pgs degraded (PG_DEGRADED) 2024-03-24T11:26:15.320 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:15.320 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:15.320 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:15.320 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:15.320 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:15.320 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:15.320 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:15.320 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:15.320 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:15.320 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:15.320 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:15.320 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:15.320 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:15.321 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:15.321 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:15.321 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:15.321 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:15.321 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:15.321 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:15.321 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:15.321 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:15.321 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:15.321 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:15.321 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:15.321 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:15.321 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:15.321 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:15.321 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:15.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:15.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:15.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:15.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:15.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:15.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:15.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:15.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:15.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:15.322 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:15.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:15.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:15.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:15.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:15.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:15.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:15.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:15.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:15.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:15.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:15.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:15.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:15.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:15.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:15.323 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:15.324 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:15 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:15.405 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:15.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:15.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:15.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:15.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:15.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:15.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:15.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:15.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:15.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:15.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:15.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:15.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:15.407 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:15.407 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:15.407 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:15.407 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:15.407 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:15.407 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:15.407 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:15.407 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:15.407 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:15.407 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:15.407 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:15.407 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:15.407 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:15.407 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:15.408 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:15.408 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:15.408 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:15.408 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:15.408 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:15.408 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:15.408 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:15.408 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:15.408 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:15.408 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:15.408 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:15.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:15.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:15.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:15.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:15.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:15.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:15.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:15.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:15.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:15.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:15.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:15.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:15.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:15.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:15.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:15.410 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:15.410 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:15.410 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:15.410 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:15.410 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:15.410 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:15.410 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:15.410 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:15.410 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:15.410 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:15.410 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:15.411 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:15.411 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:15.411 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:15.411 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:15.411 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:15.411 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:15.411 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:15.411 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:15.411 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:15.411 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:15.411 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:15.411 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:15.411 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:15.411 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:15.412 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:15.412 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:15.412 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:15.412 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:15.412 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:15.412 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:15.412 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:15.412 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:15.412 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:15.412 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:15.412 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:15.412 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:15.412 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:15.413 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:15.413 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:15.413 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:15.413 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:15.413 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:15.413 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:15.413 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:15.413 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:15.413 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:15.413 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:15.413 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:15.413 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:15.413 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:15.413 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:15.413 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:15.414 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:15.414 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:15 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:16.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: pgmap v90: 161 pgs: 161 active+clean; 585 KiB data, 116 MiB used, 715 GiB / 715 GiB avail; 837 B/s rd, 0 op/s 2024-03-24T11:26:16.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:16.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:16.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:16.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:16.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:16.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:16.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:16.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:16.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:16.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:16.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:16.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:16.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:16.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:16.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:16.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:16.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:16.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:16.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:16.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:16.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:16.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:16.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:16.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:16.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:16.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:16.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:16.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:16.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: pgmap v90: 161 pgs: 161 active+clean; 585 KiB data, 116 MiB used, 715 GiB / 715 GiB avail; 837 B/s rd, 0 op/s 2024-03-24T11:26:16.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:16.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:16.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:16.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:16.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:16.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:16.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:16.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:16.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:16.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:16.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:16.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:16.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:16.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:16.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:16.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:16.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:16.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:16.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:16.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:16.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:16.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:16.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:16.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:16.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:16.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:16.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:16.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:16.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:16.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:16.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:16.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:16.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:16.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:16.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:16.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:16.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:16.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:16.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:16.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:16.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:16.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:16.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:16.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:16.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:16.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:16.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:16.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:16.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:16.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:16.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:16.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:16.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:16.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:16.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:16.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:16.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:16.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:16.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:16.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:16.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:16.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 28/630 objects degraded (4.444%), 7 pgs degraded) 2024-03-24T11:26:16.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: Cluster is now healthy 2024-03-24T11:26:16.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:16.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:16.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:16.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:16.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:16.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:16.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:16.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:16.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:16.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:16.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:16.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:16.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:16.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:16.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:16.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:16.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:16.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:16.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:16.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:16.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:16.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:16.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:16.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:16.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:16.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:16.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:16.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:16.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:16.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:16.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:16.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:16.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:16.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:16.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:16.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:16.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:16.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:16.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 28/630 objects degraded (4.444%), 7 pgs degraded) 2024-03-24T11:26:16.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: Cluster is now healthy 2024-03-24T11:26:16.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:16.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:16.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:16.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:16.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:16.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: pgmap v90: 161 pgs: 161 active+clean; 585 KiB data, 116 MiB used, 715 GiB / 715 GiB avail; 837 B/s rd, 0 op/s 2024-03-24T11:26:16.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:16.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:16.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:16.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:16.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:16.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:16.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:16.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:16.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:16.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:16.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:16.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:16.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:16.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:16.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:16.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:16.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:16.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:16.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:16.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:16.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:16.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:16.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:16.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:16.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:16.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:16.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:16.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:16.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:16.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:16.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:16.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:16.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:16.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:16.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:16.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:16.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:16.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:16.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:16.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:16.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:16.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:16.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:16.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:16.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:16.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:16.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:16.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:16.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:16.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:16.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:16.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:16.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:16.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:16.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:16.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:16.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:16.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:16.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:16.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:16.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:16.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 28/630 objects degraded (4.444%), 7 pgs degraded) 2024-03-24T11:26:16.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: Cluster is now healthy 2024-03-24T11:26:16.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:16.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:16.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:16.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:16.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:17.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:17.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:17.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:17.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:17.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:17.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:17.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:17.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:17.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:17.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:17.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:17.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:17.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:17.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:17.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:17.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:17.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:17.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:17.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:17.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:17.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:17.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:17.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:17.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:17.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:17.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:17.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:17.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:17.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:17.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:17.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:17.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:17.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:17.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:17.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:17.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:17.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:17.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:17.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:17.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:17.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:17.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:17.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:17.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:17.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:17.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:17.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:17.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:17.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:17.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:17.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:17.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:17.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:17.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:17.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:17.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:17.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:17.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:17.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:17.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:17.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:17.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:17.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:17.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:17.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:17.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:17.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:17.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:17.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:17.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:17.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:17.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:17.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:17.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:17.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:17.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:17.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:17.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:17.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:17.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:17.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:17.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:17.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:17.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:17.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:17.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:17.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:17.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:17.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:17.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:17.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:17.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:17.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:17.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:17.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:17.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:17.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:17.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:17.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:17.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:17.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:17.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:17.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:17.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:17.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:17.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:17.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:17.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:17.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:17.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:17.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:17.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:17.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:17.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:17.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:17.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:17.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:17.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:17.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:17.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:17.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:17.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:17.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:17.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:17.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:17.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:17.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:17.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:17.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:17.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:17.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:17.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:17.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:17.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:17.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:17.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:17.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:17.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:17.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:17.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:17.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:17.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:17.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:17.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:17.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:17.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:17.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:17.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:17.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:17.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:17.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:17.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:17.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:17.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:17.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:17.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:17.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:17.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:17.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:17.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:17.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:17.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:17.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:17.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:17.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:17.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:17.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:17.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:17.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:17.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:17.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:17.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:17.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:17.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:17.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:17.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:17.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:17.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:17.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:17.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:18.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:18 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:18.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:18 smithi062 ceph-mon[98643]: pgmap v91: 161 pgs: 161 active+clean; 585 KiB data, 116 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:26:18.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:18.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:18.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:18.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:18.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:18.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:18.305 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:18.305 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:18.305 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:18.305 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:18.305 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:18.305 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:18.305 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-24T11:26:18.305 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:18 smithi062 ceph-mon[98643]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-24T11:26:18.306 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:18 smithi062 ceph-mon[98643]: Upgrade: osd.1 is safe to restart 2024-03-24T11:26:18.306 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:18.306 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:18.306 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:18.306 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:18 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:18.306 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:18 smithi062 ceph-mon[90553]: pgmap v91: 161 pgs: 161 active+clean; 585 KiB data, 116 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:26:18.306 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:18.306 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:18.306 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:18.306 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:18.307 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:18.307 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:18.307 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:18.307 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:18.307 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:18.307 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:18.307 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:18.307 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:18.307 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-24T11:26:18.307 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:18 smithi062 ceph-mon[90553]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-24T11:26:18.307 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:18 smithi062 ceph-mon[90553]: Upgrade: osd.1 is safe to restart 2024-03-24T11:26:18.307 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:18.307 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:18.307 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:18.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:18 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:18.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:18 smithi139 ceph-mon[91894]: pgmap v91: 161 pgs: 161 active+clean; 585 KiB data, 116 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:26:18.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:18.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:18.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:18.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:18.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:18.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:18.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:18.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:18.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:18.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:18.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:18.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:18.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-24T11:26:18.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:18 smithi139 ceph-mon[91894]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-24T11:26:18.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:18 smithi139 ceph-mon[91894]: Upgrade: osd.1 is safe to restart 2024-03-24T11:26:18.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:18.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:18.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:20.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:20.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:20.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:20.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:20 smithi062 ceph-mon[90553]: pgmap v92: 161 pgs: 161 active+clean; 585 KiB data, 116 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:26:20.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:20 smithi062 ceph-mon[90553]: Upgrade: Updating osd.1 2024-03-24T11:26:20.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:20.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-24T11:26:20.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:20.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:20 smithi062 ceph-mon[90553]: Deploying daemon osd.1 on smithi062 2024-03-24T11:26:20.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:20.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:20.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:20.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:20 smithi062 ceph-mon[98643]: pgmap v92: 161 pgs: 161 active+clean; 585 KiB data, 116 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:26:20.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:20 smithi062 ceph-mon[98643]: Upgrade: Updating osd.1 2024-03-24T11:26:20.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:20.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-24T11:26:20.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:20.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:20 smithi062 ceph-mon[98643]: Deploying daemon osd.1 on smithi062 2024-03-24T11:26:20.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:20.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:20.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:20.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:20 smithi139 ceph-mon[91894]: pgmap v92: 161 pgs: 161 active+clean; 585 KiB data, 116 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:26:20.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:20 smithi139 ceph-mon[91894]: Upgrade: Updating osd.1 2024-03-24T11:26:20.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:20.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-24T11:26:20.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:20.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:20 smithi139 ceph-mon[91894]: Deploying daemon osd.1 on smithi062 2024-03-24T11:26:20.977 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:26:20 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:26:20] "GET /metrics HTTP/1.1" 200 38347 "" "Prometheus/2.33.4" 2024-03-24T11:26:22.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:22 smithi062 ceph-mon[98643]: pgmap v93: 161 pgs: 161 active+clean; 585 KiB data, 116 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T11:26:22.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:22.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:22 smithi062 ceph-mon[90553]: pgmap v93: 161 pgs: 161 active+clean; 585 KiB data, 116 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T11:26:22.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:22.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:22 smithi139 ceph-mon[91894]: pgmap v93: 161 pgs: 161 active+clean; 585 KiB data, 116 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T11:26:22.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:23.322 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:23 smithi062 ceph-mon[98643]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:26:23.323 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:23 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:26:23.607 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:23 smithi139 ceph-mon[91894]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:26:23.680 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:23 smithi062 systemd[1]: Stopping Ceph osd.1 for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:26:23.702 INFO:teuthology.orchestra.run.smithi062.stdout:true 2024-03-24T11:26:23.976 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:23 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1[35856]: 2024-03-24T11:26:23.677+0000 7f56225d8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T11:26:23.977 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:23 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1[35856]: 2024-03-24T11:26:23.677+0000 7f56225d8700 -1 osd.1 118 *** Got signal Terminated *** 2024-03-24T11:26:23.977 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:23 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1[35856]: 2024-03-24T11:26:23.677+0000 7f56225d8700 -1 osd.1 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T11:26:24.400 INFO:teuthology.orchestra.run.smithi062.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T11:26:24.400 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi062 smithi062 running 6s ago 15m - - 2024-03-24T11:26:24.400 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi139 smithi139 running 5s ago 15m - - 2024-03-24T11:26:24.401 INFO:teuthology.orchestra.run.smithi062.stdout:alertmanager.a smithi062 *:9093,9094 running (5m) 6s ago 10m 24.4M - 0.23.0 ba2b418f427c 818298d58140 2024-03-24T11:26:24.401 INFO:teuthology.orchestra.run.smithi062.stdout:grafana.a smithi139 *:3000 running (4m) 5s ago 10m 53.8M - 8.3.5 dad864ee21e9 b48167e7171e 2024-03-24T11:26:24.401 INFO:teuthology.orchestra.run.smithi062.stdout:iscsi.foo.smithi062.dggsvk smithi062 running (3m) 6s ago 9m 57.0M - 3.5 e1d6a67b021e 0a98806ec839 2024-03-24T11:26:24.401 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.x smithi139 *:8443,9283,8765 running (3m) 5s ago 15m 450M - 19.0.0-1667-gdb0330b1 4f325f8440e9 d94987b11387 2024-03-24T11:26:24.401 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.y smithi062 *:8443,9283,8765 running (4m) 6s ago 16m 516M - 19.0.0-1667-gdb0330b1 4f325f8440e9 ca1a49028d65 2024-03-24T11:26:24.401 INFO:teuthology.orchestra.run.smithi062.stdout:mon.a smithi062 running (2m) 6s ago 16m 53.1M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 f7afe6e28c88 2024-03-24T11:26:24.401 INFO:teuthology.orchestra.run.smithi062.stdout:mon.b smithi139 running (90s) 5s ago 15m 39.4M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 ca357beea4dd 2024-03-24T11:26:24.401 INFO:teuthology.orchestra.run.smithi062.stdout:mon.c smithi062 running (112s) 6s ago 15m 46.6M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 459e7449ec39 2024-03-24T11:26:24.401 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.a smithi062 *:9100 running (5m) 6s ago 11m 17.3M - 1.3.1 1dbe0e931976 79d5a02a999f 2024-03-24T11:26:24.401 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.b smithi139 *:9100 running (5m) 5s ago 11m 15.7M - 1.3.1 1dbe0e931976 8f9f4608e465 2024-03-24T11:26:24.401 INFO:teuthology.orchestra.run.smithi062.stdout:osd.0 smithi062 running (19s) 6s ago 14m 49.0M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 dfe4920b49c4 2024-03-24T11:26:24.401 INFO:teuthology.orchestra.run.smithi062.stdout:osd.1 smithi062 running (14m) 6s ago 14m 58.1M 4096M 17.2.0 e1d6a67b021e e62d22e5dc29 2024-03-24T11:26:24.401 INFO:teuthology.orchestra.run.smithi062.stdout:osd.2 smithi062 running (14m) 6s ago 13m 51.6M 4096M 17.2.0 e1d6a67b021e 714f550103a7 2024-03-24T11:26:24.401 INFO:teuthology.orchestra.run.smithi062.stdout:osd.3 smithi062 running (13m) 6s ago 13m 56.7M 4096M 17.2.0 e1d6a67b021e 1d2868bd5195 2024-03-24T11:26:24.402 INFO:teuthology.orchestra.run.smithi062.stdout:osd.4 smithi139 running (13m) 5s ago 13m 60.1M 4096M 17.2.0 e1d6a67b021e 7568bfa9e74d 2024-03-24T11:26:24.402 INFO:teuthology.orchestra.run.smithi062.stdout:osd.5 smithi139 running (12m) 5s ago 12m 53.8M 4096M 17.2.0 e1d6a67b021e dd0abd599a14 2024-03-24T11:26:24.402 INFO:teuthology.orchestra.run.smithi062.stdout:osd.6 smithi139 running (12m) 5s ago 12m 53.4M 4096M 17.2.0 e1d6a67b021e a4b1289aca69 2024-03-24T11:26:24.402 INFO:teuthology.orchestra.run.smithi062.stdout:osd.7 smithi139 running (12m) 5s ago 12m 56.7M 4096M 17.2.0 e1d6a67b021e bc7fadbc1384 2024-03-24T11:26:24.402 INFO:teuthology.orchestra.run.smithi062.stdout:prometheus.a smithi139 *:9095 running (4m) 5s ago 11m 71.1M - 2.33.4 514e6a882f6e 2ed47f810d33 2024-03-24T11:26:24.402 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi062.sacpds smithi062 *:8000 running (9m) 6s ago 9m 73.5M - 17.2.0 e1d6a67b021e 89f4ac308e98 2024-03-24T11:26:24.402 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi139.yljesh smithi139 *:8000 running (9m) 5s ago 9m 72.4M - 17.2.0 e1d6a67b021e e189bdc12cae 2024-03-24T11:26:24.402 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi062.xvpxnk smithi062 *:80 running (9m) 6s ago 9m 73.3M - 17.2.0 e1d6a67b021e d1aa74adab44 2024-03-24T11:26:24.402 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi139.pbtrmp smithi139 *:80 running (9m) 5s ago 9m 76.0M - 17.2.0 e1d6a67b021e 7f8e336755d7 2024-03-24T11:26:24.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:24 smithi062 ceph-mon[90553]: pgmap v94: 161 pgs: 161 active+clean; 585 KiB data, 116 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T11:26:24.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:24 smithi062 ceph-mon[90553]: osd.1 marked itself down and dead 2024-03-24T11:26:24.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:24 smithi062 ceph-mon[90553]: from='client.64222 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:26:24.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:24 smithi062 ceph-mon[98643]: pgmap v94: 161 pgs: 161 active+clean; 585 KiB data, 116 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T11:26:24.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:24 smithi062 ceph-mon[98643]: osd.1 marked itself down and dead 2024-03-24T11:26:24.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:24 smithi062 ceph-mon[98643]: from='client.64222 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:26:24.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:24 smithi139 ceph-mon[91894]: pgmap v94: 161 pgs: 161 active+clean; 585 KiB data, 116 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T11:26:24.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:24 smithi139 ceph-mon[91894]: osd.1 marked itself down and dead 2024-03-24T11:26:24.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:24 smithi139 ceph-mon[91894]: from='client.64222 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:26:24.815 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:26:24.816 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": { 2024-03-24T11:26:24.816 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 3 2024-03-24T11:26:24.816 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:26:24.816 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": { 2024-03-24T11:26:24.816 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-24T11:26:24.816 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:26:24.816 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": { 2024-03-24T11:26:24.816 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 6, 2024-03-24T11:26:24.816 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 1 2024-03-24T11:26:24.816 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:26:24.817 INFO:teuthology.orchestra.run.smithi062.stdout: "rgw": { 2024-03-24T11:26:24.817 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4 2024-03-24T11:26:24.817 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:26:24.817 INFO:teuthology.orchestra.run.smithi062.stdout: "overall": { 2024-03-24T11:26:24.817 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 10, 2024-03-24T11:26:24.817 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 6 2024-03-24T11:26:24.817 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-24T11:26:24.817 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:26:24.956 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:24 smithi062 podman[119348]: 2024-03-24 11:26:24.678311581 +0000 UTC m=+1.096838856 container died e62d22e5dc29fadb744597defdd5fc1d05cb2e8f239c0d9c29cadc68a2b64211 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1, version=8, architecture=x86_64, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, maintainer=Guillaume Abrioux , io.buildah.version=1.19.8, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, vcs-type=git, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, distribution-scope=public, com.redhat.component=centos-stream-container, RELEASE=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, release=754, io.openshift.tags=base centos centos-stream, GIT_CLEAN=True, name=centos-stream, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, io.openshift.expose-services=, vendor=Red Hat, Inc., GIT_BRANCH=HEAD, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, ceph=True, build-date=2022-05-03T08:36:31.336870, io.k8s.display-name=CentOS Stream 8, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, CEPH_POINT_RELEASE=-17.2.0) 2024-03-24T11:26:25.205 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:26:25.205 INFO:teuthology.orchestra.run.smithi062.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69", 2024-03-24T11:26:25.205 INFO:teuthology.orchestra.run.smithi062.stdout: "in_progress": true, 2024-03-24T11:26:25.205 INFO:teuthology.orchestra.run.smithi062.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T11:26:25.205 INFO:teuthology.orchestra.run.smithi062.stdout: "services_complete": [ 2024-03-24T11:26:25.205 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr", 2024-03-24T11:26:25.205 INFO:teuthology.orchestra.run.smithi062.stdout: "mon" 2024-03-24T11:26:25.206 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-03-24T11:26:25.206 INFO:teuthology.orchestra.run.smithi062.stdout: "progress": "6/23 daemons upgraded", 2024-03-24T11:26:25.206 INFO:teuthology.orchestra.run.smithi062.stdout: "message": "Currently upgrading osd daemons", 2024-03-24T11:26:25.206 INFO:teuthology.orchestra.run.smithi062.stdout: "is_paused": false 2024-03-24T11:26:25.206 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:26:25.227 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:25 smithi062 podman[119348]: 2024-03-24 11:26:25.063286475 +0000 UTC m=+1.481813732 container cleanup e62d22e5dc29fadb744597defdd5fc1d05cb2e8f239c0d9c29cadc68a2b64211 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1, vcs-type=git, version=8, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, architecture=x86_64, GIT_CLEAN=True, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, io.openshift.tags=base centos centos-stream, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, CEPH_POINT_RELEASE=-17.2.0, RELEASE=HEAD, com.redhat.component=centos-stream-container, maintainer=Guillaume Abrioux , ceph=True, io.k8s.display-name=CentOS Stream 8, io.openshift.expose-services=, GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.19.8, release=754, GIT_BRANCH=HEAD, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., distribution-scope=public, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, name=centos-stream, build-date=2022-05-03T08:36:31.336870, vendor=Red Hat, Inc., url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac) 2024-03-24T11:26:25.227 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:25 smithi062 bash[119348]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1 2024-03-24T11:26:25.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:25 smithi062 ceph-mon[98643]: from='client.44262 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:26:25.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:25 smithi062 ceph-mon[98643]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T11:26:25.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:25 smithi062 ceph-mon[98643]: failed to encode map e119 with expected crc 2024-03-24T11:26:25.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:25 smithi062 ceph-mon[98643]: failed to encode map e119 with expected crc 2024-03-24T11:26:25.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:25 smithi062 ceph-mon[98643]: osdmap e119: 8 total, 7 up, 8 in 2024-03-24T11:26:25.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:25 smithi062 ceph-mon[98643]: failed to encode map e119 with expected crc 2024-03-24T11:26:25.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:25 smithi062 ceph-mon[98643]: failed to encode map e119 with expected crc 2024-03-24T11:26:25.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:25 smithi062 ceph-mon[98643]: failed to encode map e119 with expected crc 2024-03-24T11:26:25.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:25 smithi062 ceph-mon[98643]: failed to encode map e119 with expected crc 2024-03-24T11:26:25.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:25 smithi062 ceph-mon[98643]: failed to encode map e119 with expected crc 2024-03-24T11:26:25.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:25 smithi062 ceph-mon[98643]: failed to encode map e119 with expected crc 2024-03-24T11:26:25.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:25 smithi062 ceph-mon[98643]: from='client.44268 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:26:25.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:25 smithi062 ceph-mon[98643]: from='client.? 172.21.15.62:0/3274759578' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:25.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:25 smithi062 ceph-mon[90553]: from='client.44262 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:26:25.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:25 smithi062 ceph-mon[90553]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T11:26:25.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:25 smithi062 ceph-mon[90553]: failed to encode map e119 with expected crc 2024-03-24T11:26:25.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:25 smithi062 ceph-mon[90553]: failed to encode map e119 with expected crc 2024-03-24T11:26:25.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:25 smithi062 ceph-mon[90553]: osdmap e119: 8 total, 7 up, 8 in 2024-03-24T11:26:25.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:25 smithi062 ceph-mon[90553]: failed to encode map e119 with expected crc 2024-03-24T11:26:25.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:25 smithi062 ceph-mon[90553]: failed to encode map e119 with expected crc 2024-03-24T11:26:25.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:25 smithi062 ceph-mon[90553]: failed to encode map e119 with expected crc 2024-03-24T11:26:25.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:25 smithi062 ceph-mon[90553]: failed to encode map e119 with expected crc 2024-03-24T11:26:25.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:25 smithi062 ceph-mon[90553]: failed to encode map e119 with expected crc 2024-03-24T11:26:25.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:25 smithi062 ceph-mon[90553]: failed to encode map e119 with expected crc 2024-03-24T11:26:25.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:25 smithi062 ceph-mon[90553]: from='client.44268 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:26:25.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:25 smithi062 ceph-mon[90553]: from='client.? 172.21.15.62:0/3274759578' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:25.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:25 smithi139 ceph-mon[91894]: from='client.44262 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:26:25.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:25 smithi139 ceph-mon[91894]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T11:26:25.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:25 smithi139 ceph-mon[91894]: failed to encode map e119 with expected crc 2024-03-24T11:26:25.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:25 smithi139 ceph-mon[91894]: failed to encode map e119 with expected crc 2024-03-24T11:26:25.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:25 smithi139 ceph-mon[91894]: osdmap e119: 8 total, 7 up, 8 in 2024-03-24T11:26:25.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:25 smithi139 ceph-mon[91894]: failed to encode map e119 with expected crc 2024-03-24T11:26:25.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:25 smithi139 ceph-mon[91894]: failed to encode map e119 with expected crc 2024-03-24T11:26:25.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:25 smithi139 ceph-mon[91894]: failed to encode map e119 with expected crc 2024-03-24T11:26:25.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:25 smithi139 ceph-mon[91894]: failed to encode map e119 with expected crc 2024-03-24T11:26:25.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:25 smithi139 ceph-mon[91894]: failed to encode map e119 with expected crc 2024-03-24T11:26:25.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:25 smithi139 ceph-mon[91894]: failed to encode map e119 with expected crc 2024-03-24T11:26:25.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:25 smithi139 ceph-mon[91894]: from='client.44268 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:26:25.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:25 smithi139 ceph-mon[91894]: from='client.? 172.21.15.62:0/3274759578' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:25.655 INFO:teuthology.orchestra.run.smithi062.stdout:HEALTH_WARN 1 osds down 2024-03-24T11:26:25.656 INFO:teuthology.orchestra.run.smithi062.stdout:[WRN] OSD_DOWN: 1 osds down 2024-03-24T11:26:25.656 INFO:teuthology.orchestra.run.smithi062.stdout: osd.1 (root=default,host=smithi062) is down 2024-03-24T11:26:25.656 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:25 smithi062 podman[119434]: 2024-03-24 11:26:25.340085564 +0000 UTC m=+0.658216829 container remove e62d22e5dc29fadb744597defdd5fc1d05cb2e8f239c0d9c29cadc68a2b64211 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1, version=8, io.buildah.version=1.19.8, ceph=True, build-date=2022-05-03T08:36:31.336870, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, io.openshift.tags=base centos centos-stream, io.k8s.display-name=CentOS Stream 8, RELEASE=HEAD, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, vcs-type=git, GIT_REPO=https://github.com/ceph/ceph-container.git, distribution-scope=public, CEPH_POINT_RELEASE=-17.2.0, io.openshift.expose-services=, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, maintainer=Guillaume Abrioux , vendor=Red Hat, Inc., description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, com.redhat.component=centos-stream-container, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, GIT_CLEAN=True, GIT_BRANCH=HEAD, architecture=x86_64, release=754, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., name=centos-stream, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754) 2024-03-24T11:26:25.977 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:25 smithi062 podman[119573]: 2024-03-24 11:26:25.580830247 +0000 UTC m=+0.017661228 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:26:25.977 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:25 smithi062 podman[119573]: 2024-03-24 11:26:25.700041676 +0000 UTC m=+0.136872691 container create 7b6927153df387994f50a7657d86f0dd596a9f3cc5e47a2beab660c0db0213d4 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T11:26:25.977 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:25 smithi062 podman[119573]: 2024-03-24 11:26:25.877377429 +0000 UTC m=+0.314208414 container init 7b6927153df387994f50a7657d86f0dd596a9f3cc5e47a2beab660c0db0213d4 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-db0330b, org.label-schema.license=GPLv2) 2024-03-24T11:26:25.977 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:25 smithi062 podman[119573]: 2024-03-24 11:26:25.880574067 +0000 UTC m=+0.317405047 container start 7b6927153df387994f50a7657d86f0dd596a9f3cc5e47a2beab660c0db0213d4 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240307, RELEASE=squid-db0330b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-24T11:26:26.456 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:25 smithi062 podman[119573]: 2024-03-24 11:26:25.977517637 +0000 UTC m=+0.414348644 container attach 7b6927153df387994f50a7657d86f0dd596a9f3cc5e47a2beab660c0db0213d4 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-db0330b, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-24T11:26:26.456 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:26 smithi062 podman[119573]: 2024-03-24 11:26:26.177692345 +0000 UTC m=+0.614523319 container died 7b6927153df387994f50a7657d86f0dd596a9f3cc5e47a2beab660c0db0213d4 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-db0330b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-24T11:26:26.457 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:26 smithi062 ceph-mon[90553]: pgmap v96: 161 pgs: 19 stale+active+clean, 142 active+clean; 585 KiB data, 116 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T11:26:26.457 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:26 smithi062 ceph-mon[90553]: from='client.44280 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:26:26.457 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:26 smithi062 ceph-mon[90553]: failed to encode map e120 with expected crc 2024-03-24T11:26:26.457 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:26 smithi062 ceph-mon[90553]: failed to encode map e120 with expected crc 2024-03-24T11:26:26.457 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:26 smithi062 ceph-mon[90553]: osdmap e120: 8 total, 7 up, 8 in 2024-03-24T11:26:26.457 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:26 smithi062 ceph-mon[90553]: failed to encode map e120 with expected crc 2024-03-24T11:26:26.457 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:26 smithi062 ceph-mon[90553]: failed to encode map e120 with expected crc 2024-03-24T11:26:26.457 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:26 smithi062 ceph-mon[90553]: failed to encode map e120 with expected crc 2024-03-24T11:26:26.457 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:26 smithi062 ceph-mon[90553]: failed to encode map e120 with expected crc 2024-03-24T11:26:26.457 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:26 smithi062 ceph-mon[90553]: failed to encode map e120 with expected crc 2024-03-24T11:26:26.457 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:26 smithi062 ceph-mon[90553]: failed to encode map e120 with expected crc 2024-03-24T11:26:26.457 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:26 smithi062 ceph-mon[90553]: from='client.? 172.21.15.62:0/3612038251' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:26:26.457 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:26.457 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:26:26.458 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:26 smithi062 ceph-mon[98643]: pgmap v96: 161 pgs: 19 stale+active+clean, 142 active+clean; 585 KiB data, 116 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T11:26:26.458 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:26 smithi062 ceph-mon[98643]: from='client.44280 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:26:26.458 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:26 smithi062 ceph-mon[98643]: failed to encode map e120 with expected crc 2024-03-24T11:26:26.458 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:26 smithi062 ceph-mon[98643]: failed to encode map e120 with expected crc 2024-03-24T11:26:26.458 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:26 smithi062 ceph-mon[98643]: osdmap e120: 8 total, 7 up, 8 in 2024-03-24T11:26:26.458 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:26 smithi062 ceph-mon[98643]: failed to encode map e120 with expected crc 2024-03-24T11:26:26.458 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:26 smithi062 ceph-mon[98643]: failed to encode map e120 with expected crc 2024-03-24T11:26:26.458 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:26 smithi062 ceph-mon[98643]: failed to encode map e120 with expected crc 2024-03-24T11:26:26.458 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:26 smithi062 ceph-mon[98643]: failed to encode map e120 with expected crc 2024-03-24T11:26:26.458 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:26 smithi062 ceph-mon[98643]: failed to encode map e120 with expected crc 2024-03-24T11:26:26.458 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:26 smithi062 ceph-mon[98643]: failed to encode map e120 with expected crc 2024-03-24T11:26:26.458 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:26 smithi062 ceph-mon[98643]: from='client.? 172.21.15.62:0/3612038251' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:26:26.458 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:26.459 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:26:26.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:26 smithi139 ceph-mon[91894]: pgmap v96: 161 pgs: 19 stale+active+clean, 142 active+clean; 585 KiB data, 116 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T11:26:26.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:26 smithi139 ceph-mon[91894]: from='client.44280 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:26:26.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:26 smithi139 ceph-mon[91894]: failed to encode map e120 with expected crc 2024-03-24T11:26:26.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:26 smithi139 ceph-mon[91894]: failed to encode map e120 with expected crc 2024-03-24T11:26:26.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:26 smithi139 ceph-mon[91894]: osdmap e120: 8 total, 7 up, 8 in 2024-03-24T11:26:26.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:26 smithi139 ceph-mon[91894]: failed to encode map e120 with expected crc 2024-03-24T11:26:26.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:26 smithi139 ceph-mon[91894]: failed to encode map e120 with expected crc 2024-03-24T11:26:26.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:26 smithi139 ceph-mon[91894]: failed to encode map e120 with expected crc 2024-03-24T11:26:26.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:26 smithi139 ceph-mon[91894]: failed to encode map e120 with expected crc 2024-03-24T11:26:26.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:26 smithi139 ceph-mon[91894]: failed to encode map e120 with expected crc 2024-03-24T11:26:26.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:26 smithi139 ceph-mon[91894]: failed to encode map e120 with expected crc 2024-03-24T11:26:26.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:26 smithi139 ceph-mon[91894]: from='client.? 172.21.15.62:0/3612038251' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:26:26.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:26.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:26:27.068 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:26 smithi062 podman[119602]: 2024-03-24 11:26:26.815163472 +0000 UTC m=+0.628275219 container remove 7b6927153df387994f50a7657d86f0dd596a9f3cc5e47a2beab660c0db0213d4 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, RELEASE=squid-db0330b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-24T11:26:27.068 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:26 smithi062 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.1.service: Deactivated successfully. 2024-03-24T11:26:27.068 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:26 smithi062 systemd[1]: Stopped Ceph osd.1 for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:26:27.068 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:26 smithi062 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.1.service: Consumed 8.542s CPU time. 2024-03-24T11:26:27.068 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:27 smithi062 systemd[1]: Starting Ceph osd.1 for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:26:27.424 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:27 smithi062 ceph-mon[90553]: Health check failed: Degraded data redundancy: 83/630 objects degraded (13.175%), 22 pgs degraded (PG_DEGRADED) 2024-03-24T11:26:27.424 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:27 smithi062 ceph-mon[98643]: Health check failed: Degraded data redundancy: 83/630 objects degraded (13.175%), 22 pgs degraded (PG_DEGRADED) 2024-03-24T11:26:27.424 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:27 smithi062 podman[119703]: 2024-03-24 11:26:27.18670275 +0000 UTC m=+0.017814010 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:26:27.424 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:27 smithi062 podman[119703]: 2024-03-24 11:26:27.317651161 +0000 UTC m=+0.148762454 container create 0481cb165ab2051aed578d29c23dd304d74753eae9d659dacd5ee67911be7eb7 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1-activate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-24T11:26:27.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:27 smithi139 ceph-mon[91894]: Health check failed: Degraded data redundancy: 83/630 objects degraded (13.175%), 22 pgs degraded (PG_DEGRADED) 2024-03-24T11:26:27.727 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:27 smithi062 podman[119703]: 2024-03-24 11:26:27.519355561 +0000 UTC m=+0.350466821 container init 0481cb165ab2051aed578d29c23dd304d74753eae9d659dacd5ee67911be7eb7 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1-activate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-db0330b, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-03-24T11:26:27.727 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:27 smithi062 podman[119703]: 2024-03-24 11:26:27.522628142 +0000 UTC m=+0.353739401 container start 0481cb165ab2051aed578d29c23dd304d74753eae9d659dacd5ee67911be7eb7 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1-activate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-24T11:26:27.727 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:27 smithi062 podman[119703]: 2024-03-24 11:26:27.602632871 +0000 UTC m=+0.433744143 container attach 0481cb165ab2051aed578d29c23dd304d74753eae9d659dacd5ee67911be7eb7 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-24T11:26:28.227 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:27 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1-activate[119719]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-24T11:26:28.227 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:27 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1-activate[119719]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-24T11:26:28.227 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:27 smithi062 bash[119703]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-24T11:26:28.227 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:27 smithi062 bash[119703]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-24T11:26:28.227 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:27 smithi062 bash[119703]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-24T11:26:28.227 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:27 smithi062 bash[119703]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-24T11:26:28.228 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:27 smithi062 bash[119703]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-03-24T11:26:28.228 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:27 smithi062 bash[119703]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-24T11:26:28.228 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:27 smithi062 bash[119703]: --> ceph-volume raw activate successful for osd ID: 1 2024-03-24T11:26:28.228 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:27 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1-activate[119719]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-24T11:26:28.228 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:27 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1-activate[119719]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-24T11:26:28.228 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:27 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1-activate[119719]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-03-24T11:26:28.228 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:27 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1-activate[119719]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-24T11:26:28.228 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:27 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1-activate[119719]: --> ceph-volume raw activate successful for osd ID: 1 2024-03-24T11:26:28.228 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:27 smithi062 podman[119703]: 2024-03-24 11:26:27.994705689 +0000 UTC m=+0.825816943 container died 0481cb165ab2051aed578d29c23dd304d74753eae9d659dacd5ee67911be7eb7 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1-activate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-db0330b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-24T11:26:28.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:28 smithi139 ceph-mon[91894]: pgmap v98: 161 pgs: 22 active+undersized+degraded, 1 stale+active+clean, 34 active+undersized, 104 active+clean; 585 KiB data, 117 MiB used, 715 GiB / 715 GiB avail; 83/630 objects degraded (13.175%) 2024-03-24T11:26:28.640 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:28 smithi062 ceph-mon[90553]: pgmap v98: 161 pgs: 22 active+undersized+degraded, 1 stale+active+clean, 34 active+undersized, 104 active+clean; 585 KiB data, 117 MiB used, 715 GiB / 715 GiB avail; 83/630 objects degraded (13.175%) 2024-03-24T11:26:28.641 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:28 smithi062 ceph-mon[98643]: pgmap v98: 161 pgs: 22 active+undersized+degraded, 1 stale+active+clean, 34 active+undersized, 104 active+clean; 585 KiB data, 117 MiB used, 715 GiB / 715 GiB avail; 83/630 objects degraded (13.175%) 2024-03-24T11:26:28.977 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:28 smithi062 podman[119841]: 2024-03-24 11:26:28.640522385 +0000 UTC m=+0.640941144 container remove 0481cb165ab2051aed578d29c23dd304d74753eae9d659dacd5ee67911be7eb7 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1-activate, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-24T11:26:28.977 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:28 smithi062 podman[119913]: 2024-03-24 11:26:28.766063579 +0000 UTC m=+0.018486203 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:26:28.977 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:28 smithi062 podman[119913]: 2024-03-24 11:26:28.883425084 +0000 UTC m=+0.135847701 container create 51ec6bd391419265e5df614fc4db6829b253fbcfb5f9f70e90d68a6d3f01aa5e (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, RELEASE=squid-db0330b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True) 2024-03-24T11:26:29.435 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:29 smithi062 podman[119913]: 2024-03-24 11:26:29.11934452 +0000 UTC m=+0.371767149 container init 51ec6bd391419265e5df614fc4db6829b253fbcfb5f9f70e90d68a6d3f01aa5e (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1, RELEASE=squid-db0330b, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True) 2024-03-24T11:26:29.435 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:29 smithi062 podman[119913]: 2024-03-24 11:26:29.122793348 +0000 UTC m=+0.375215965 container start 51ec6bd391419265e5df614fc4db6829b253fbcfb5f9f70e90d68a6d3f01aa5e (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-db0330b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-24T11:26:29.435 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:29 smithi062 ceph-osd[119932]: -- 172.21.15.62:0/673841121 <== mon.0 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d75577b1e0 con 0x55d75574cc00 2024-03-24T11:26:29.435 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:29 smithi062 bash[119913]: 51ec6bd391419265e5df614fc4db6829b253fbcfb5f9f70e90d68a6d3f01aa5e 2024-03-24T11:26:29.435 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:29 smithi062 systemd[1]: Started Ceph osd.1 for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:26:29.698 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:29 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1[119928]: 2024-03-24T11:26:29.663+0000 7f8001b617c0 -1 Falling back to public interface 2024-03-24T11:26:30.533 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[90553]: pgmap v99: 161 pgs: 24 active+undersized+degraded, 39 active+undersized, 98 active+clean; 585 KiB data, 121 MiB used, 715 GiB / 715 GiB avail; 90/630 objects degraded (14.286%) 2024-03-24T11:26:30.533 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:30.533 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:30.533 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:30.533 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:30.533 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:30.533 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:30.533 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:30.533 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:30.533 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:30.534 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:30.534 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:30.534 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:30.534 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:30.534 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:30.534 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:30.534 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:30.534 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:30.534 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:30.534 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:30.534 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:30.534 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:30.534 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:30.534 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:30.535 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:30.535 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:30.535 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:30.535 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:30.535 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:30.535 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[98643]: pgmap v99: 161 pgs: 24 active+undersized+degraded, 39 active+undersized, 98 active+clean; 585 KiB data, 121 MiB used, 715 GiB / 715 GiB avail; 90/630 objects degraded (14.286%) 2024-03-24T11:26:30.535 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:30.535 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:30.535 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:30.535 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:30.536 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:30.536 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:30.536 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:30.536 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:30.536 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:30.536 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:30.536 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:30.536 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:30.536 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:30.536 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:30.536 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:30.536 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:30.536 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:30.536 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:30.537 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:30.537 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:30.537 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:30.537 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:30.537 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:30.537 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:30.537 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:30.537 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:30.537 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:30.537 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:30.537 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:30.538 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:30.538 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:30.538 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:30.538 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:30.538 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:30.538 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:30.538 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:30.538 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:30.538 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:30.538 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:30.538 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:30.538 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:30.538 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:30.538 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:30.539 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:30.539 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:30.539 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:30.539 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:30.540 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:30.540 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:30.540 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:30.540 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:30.540 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:30.540 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:30.540 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:30.540 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:30.540 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:30.540 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:30.540 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:30.540 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:30.540 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:30.540 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:30.540 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:30.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:30 smithi139 ceph-mon[91894]: pgmap v99: 161 pgs: 24 active+undersized+degraded, 39 active+undersized, 98 active+clean; 585 KiB data, 121 MiB used, 715 GiB / 715 GiB avail; 90/630 objects degraded (14.286%) 2024-03-24T11:26:30.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:30.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:30.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:30.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:30.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:30.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:30 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:30.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:30.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:30.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:30.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:30.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:30.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:30 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:30.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:30.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:30.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:30.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:30.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:30.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:30 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:30.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:30.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:30.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:30.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:30.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:30.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:30 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:30.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:30.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:30.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:30.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:30.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:30.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:30 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:30.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:30.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:30.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:30.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:30.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:30.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:30.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:30.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:30.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:30.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:30.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:30.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:30.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:30.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:30.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:31.164 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:26:30 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:26:30] "GET /metrics HTTP/1.1" 200 38492 "" "Prometheus/2.33.4" 2024-03-24T11:26:31.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:31.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:31.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:31.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:31.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:31.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:31.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:31.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:31.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:31.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:31.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:31.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:31.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:31.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:31.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:31.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:31.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:31.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:31.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:31.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:31.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:31.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:31.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:31.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:31.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:31.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:31.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:31.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:31.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:31.476 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:31.476 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:31.476 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:31.476 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:31.476 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:31.476 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:31.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:31.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:31.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:31.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:31.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:31.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:31.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:31.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:31.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:31.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:31.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:31.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:31.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:31.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:31.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:31.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:31.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:31.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:31.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:31.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:31.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:31.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:31.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:31.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:31.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:31.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:31.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:31.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:31.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:31.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:31.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:31.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:31.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:31.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:31.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:31.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:31.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:31.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:31.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:31.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:31.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:31.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:31.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:31.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:31.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:31.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:31.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:31.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:31.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:31.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:31.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:31.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:31.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:31.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:31.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:31.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:31.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:31.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:31.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:31.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:31.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:31.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:31.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:31.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:31.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:31.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:31.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:31.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:31.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:31.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:31.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:31.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:31.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:31.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:31.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:31.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:31.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:31.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:31.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:31.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:31.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:31.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:31.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:31.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:31.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:31.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:31.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:31.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:31.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:31.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:31.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:31.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:31.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:31.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:31.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:31.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:31.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:31.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:31.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:31.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:31.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:31.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:31.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:31.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:31.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:31.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:31.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:31.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:31.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:31.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:31.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:31.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:31.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:31.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:31.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:31.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:31.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:31.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:31.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:31.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:31.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:31.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:31.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:31.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:31.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:31.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:31.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:31.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:31.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:31.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:31.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:31.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:31.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:31.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:31.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:31.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:31.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:31.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:31.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:31.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:31.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:31.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:31.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:31.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:31.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:32.477 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1[119928]: 2024-03-24T11:26:32.398+0000 7f8001b617c0 -1 osd.1 0 read_superblock omap replica is missing. 2024-03-24T11:26:32.477 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1[119928]: 2024-03-24T11:26:32.431+0000 7f8001b617c0 -1 osd.1 118 log_to_monitors true 2024-03-24T11:26:32.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:32.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: pgmap v100: 161 pgs: 24 active+undersized+degraded, 39 active+undersized, 98 active+clean; 585 KiB data, 121 MiB used, 715 GiB / 715 GiB avail; 90/630 objects degraded (14.286%) 2024-03-24T11:26:32.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:32.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:32.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:32.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:32.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:32.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:32.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:32.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:32.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:32.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:32.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:32.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:32.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:32.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:32.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:32.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:32.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:32.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:32.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:32.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:32.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:32.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:32.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:32.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:32.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:32.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:32.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:32.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:32.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: pgmap v100: 161 pgs: 24 active+undersized+degraded, 39 active+undersized, 98 active+clean; 585 KiB data, 121 MiB used, 715 GiB / 715 GiB avail; 90/630 objects degraded (14.286%) 2024-03-24T11:26:32.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:32.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:32.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:32.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:32.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:32.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:32.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:32.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:32.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:32.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:32.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:32.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:32.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:32.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:32.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:32.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:32.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:32.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:32.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:32.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:32.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:32.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:32.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:32.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:32.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:32.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:32.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:32.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:32.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:32.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:32.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:32.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:32.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:32.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:32.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:32.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:32.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:32.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:32.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:32.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:32.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:32.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:32.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:32.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:32.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:32.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:32.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:32.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:32.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:32.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:32.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:32.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:32.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:32.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:32.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:32.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:32.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:32.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:32.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:32.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:32.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:32.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:32.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:32.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:32.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:32.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:32.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:32.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:32.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:32.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:32.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:32.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:32.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:32.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:32.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:32.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:32.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:32.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:32.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:32.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:32.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:32.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:32.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:32.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:32.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:32.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:32.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:32.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:32.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:32.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:32.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:32.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:32.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:32.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:32.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:32.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:32.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:32.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:32.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:32.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:32.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:32.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:32.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:32.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:32.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: pgmap v100: 161 pgs: 24 active+undersized+degraded, 39 active+undersized, 98 active+clean; 585 KiB data, 121 MiB used, 715 GiB / 715 GiB avail; 90/630 objects degraded (14.286%) 2024-03-24T11:26:32.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:32.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:32.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:32.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:32.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:32.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:32.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:32.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:32.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:32.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:32.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:32.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:32.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:32.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:32.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:32.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:32.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:32.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:32.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:32.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:32.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:32.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:32.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:32.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:32.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:32.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:32.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:32.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:32.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:32.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:32.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:32.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:32.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:32.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:32.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:32.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:32.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:32.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:32.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:32.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:32.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:32.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:32.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:32.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:32.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:32.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:32.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:32.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:32.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:32.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:32.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:32.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:32.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:32.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:32.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:32.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:32.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:32.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:32.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:32.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:32.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:32.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:32.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:32.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:32.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:33.476 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1[119928]: 2024-03-24T11:26:33.368+0000 7f7ff2fb9700 -1 osd.1 118 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T11:26:33.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:33.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:33.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:26:33.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:33.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:33.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:33.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:33.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:33.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:33.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:33.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:33.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:33.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:33.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:33.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: from='osd.1 [v2:172.21.15.62:6810/3930446392,v1:172.21.15.62:6811/3930446392]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-24T11:26:33.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:33.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:33.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:33.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:33.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:33.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:33.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:33.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:33.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:33.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:33.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:33.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:33.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:33.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:33.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:33.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:33.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:33.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:33.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:33.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:33.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:33.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:33.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:33.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:33.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:33.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:33.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:33.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:33.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:33.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:33.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:33.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:33.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:33.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:33.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:33.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:33.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:33.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:33.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:33.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:33.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:33.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:33.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:33.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:33.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:26:33.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:33.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:33.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:33.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:33.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:33.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:33.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:33.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:33.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:33.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:33.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:33.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: from='osd.1 [v2:172.21.15.62:6810/3930446392,v1:172.21.15.62:6811/3930446392]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-24T11:26:33.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:33.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:33.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:33.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:33.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:33.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:33.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:33.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:33.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:33.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:33.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:33.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:33.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:33.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:33.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:33.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:33.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:33.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:33.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:33.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:33.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:33.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:33.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:33.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:33.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:33.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:33.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:33.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:33.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:33.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:33.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:33.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:33.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:33.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:33.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:33.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:33.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:33.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:33.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:33.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:33.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:33.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:33.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:33.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:33.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:26:33.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:33.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:33.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:33.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:33.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:33.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:33.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:33.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:33.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:33.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:33.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:33.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: from='osd.1 [v2:172.21.15.62:6810/3930446392,v1:172.21.15.62:6811/3930446392]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-24T11:26:33.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:33.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:33.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:33.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:33.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:33.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:33.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:33.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:33.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:33.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:33.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:33.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:33.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:33.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:33.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:33.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:33.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:33.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:33.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:33.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:33.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:33.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:33.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:33.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:33.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:33.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:33.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:33.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:33.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:33.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:33.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:33.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:33.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:33.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:33.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:33.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:33.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:33.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:33.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:33.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:33.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:33.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:34.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:34.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: pgmap v101: 161 pgs: 24 active+undersized+degraded, 39 active+undersized, 98 active+clean; 585 KiB data, 121 MiB used, 715 GiB / 715 GiB avail; 90/630 objects degraded (14.286%) 2024-03-24T11:26:34.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:34.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: from='osd.1 [v2:172.21.15.62:6810/3930446392,v1:172.21.15.62:6811/3930446392]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-24T11:26:34.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: failed to encode map e121 with expected crc 2024-03-24T11:26:34.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: failed to encode map e121 with expected crc 2024-03-24T11:26:34.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: osdmap e121: 8 total, 7 up, 8 in 2024-03-24T11:26:34.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: from='osd.1 [v2:172.21.15.62:6810/3930446392,v1:172.21.15.62:6811/3930446392]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:26:34.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:34.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:34.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:34.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:34.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:34.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:34.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:34.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:34.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:34.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:34.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:34.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:34.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:34.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:34.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:34.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:34.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:34.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:34.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:34.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:34.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:34.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:34.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:34.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:34.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:34.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:34.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:34.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:34.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:34.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:34.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:34.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:34.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:34.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:34.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:34.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:34.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:34.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:34.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:34.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:34.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:34.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:34.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:34.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:34.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:34.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:34.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: Health check update: Degraded data redundancy: 90/630 objects degraded (14.286%), 24 pgs degraded (PG_DEGRADED) 2024-03-24T11:26:34.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:34.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:34.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:34.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:34.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:34.610 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:34.610 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: pgmap v101: 161 pgs: 24 active+undersized+degraded, 39 active+undersized, 98 active+clean; 585 KiB data, 121 MiB used, 715 GiB / 715 GiB avail; 90/630 objects degraded (14.286%) 2024-03-24T11:26:34.610 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:34.610 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: from='osd.1 [v2:172.21.15.62:6810/3930446392,v1:172.21.15.62:6811/3930446392]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-24T11:26:34.610 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: failed to encode map e121 with expected crc 2024-03-24T11:26:34.610 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: failed to encode map e121 with expected crc 2024-03-24T11:26:34.610 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: osdmap e121: 8 total, 7 up, 8 in 2024-03-24T11:26:34.610 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: from='osd.1 [v2:172.21.15.62:6810/3930446392,v1:172.21.15.62:6811/3930446392]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:26:34.610 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:34.611 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:34.611 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:34.611 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:34.611 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:34.611 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:34.611 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:34.611 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:34.611 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:34.611 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:34.611 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:34.611 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:34.611 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:34.611 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:34.611 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:34.612 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:34.612 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:34.612 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:34.612 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:34.612 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:34.612 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:34.612 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: pgmap v101: 161 pgs: 24 active+undersized+degraded, 39 active+undersized, 98 active+clean; 585 KiB data, 121 MiB used, 715 GiB / 715 GiB avail; 90/630 objects degraded (14.286%) 2024-03-24T11:26:34.613 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:34.613 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: from='osd.1 [v2:172.21.15.62:6810/3930446392,v1:172.21.15.62:6811/3930446392]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-24T11:26:34.613 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: failed to encode map e121 with expected crc 2024-03-24T11:26:34.613 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: failed to encode map e121 with expected crc 2024-03-24T11:26:34.613 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: osdmap e121: 8 total, 7 up, 8 in 2024-03-24T11:26:34.613 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: from='osd.1 [v2:172.21.15.62:6810/3930446392,v1:172.21.15.62:6811/3930446392]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:26:34.613 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:34.613 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:34.613 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:34.613 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:34.613 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:34.613 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:34.613 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:34.613 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:34.614 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:34.614 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:34.614 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:34.614 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:34.614 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:34.614 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:34.614 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:34.614 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:34.614 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:34.614 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:34.614 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:34.614 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:34.614 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:34.614 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:34.615 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:34.615 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:34.615 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:34.615 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:34.615 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:34.615 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:34.615 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:34.615 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:34.615 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:34.615 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:34.615 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:34.615 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:34.616 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:34.616 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:34.616 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:34.616 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:34.616 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:34.616 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:34.616 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:34.616 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:34.616 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:34.616 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:34.616 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:34.616 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:34.616 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: Health check update: Degraded data redundancy: 90/630 objects degraded (14.286%), 24 pgs degraded (PG_DEGRADED) 2024-03-24T11:26:34.616 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:34.616 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:34.616 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:34.617 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:34.617 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:34.617 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:34.617 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:34.617 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:34.617 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:34.617 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:34.617 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:34.617 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:34.617 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:34.618 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:34.618 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:34.618 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:34.618 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:34.618 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:34.618 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:34.618 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:34.618 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:34.618 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:34.618 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:34.618 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:34.618 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:34.618 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:34.618 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:34.618 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:34.619 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:34.619 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:34.619 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:34.619 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: Health check update: Degraded data redundancy: 90/630 objects degraded (14.286%), 24 pgs degraded (PG_DEGRADED) 2024-03-24T11:26:34.619 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:34.619 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:34.619 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:34.619 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:34.619 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:35.400 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:35.401 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:35.401 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:35.401 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T11:26:35.401 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: failed to encode map e122 with expected crc 2024-03-24T11:26:35.401 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: failed to encode map e122 with expected crc 2024-03-24T11:26:35.401 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: osd.1 [v2:172.21.15.62:6810/3930446392,v1:172.21.15.62:6811/3930446392] boot 2024-03-24T11:26:35.401 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: osdmap e122: 8 total, 8 up, 8 in 2024-03-24T11:26:35.401 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T11:26:35.401 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: failed to encode map e122 with expected crc 2024-03-24T11:26:35.401 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: failed to encode map e122 with expected crc 2024-03-24T11:26:35.401 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: failed to encode map e122 with expected crc 2024-03-24T11:26:35.401 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: failed to encode map e122 with expected crc 2024-03-24T11:26:35.401 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: failed to encode map e122 with expected crc 2024-03-24T11:26:35.401 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:35.401 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:35.402 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:35.402 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:35.402 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:35.402 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:35.402 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:35.402 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:35.402 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:35.402 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:35.402 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:35.402 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:35.402 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:35.402 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:35.402 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:35.402 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:35.403 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:35.403 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:35.403 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:35.403 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:35.403 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:35.403 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:35.403 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:35.403 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:35.403 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:35.404 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:35.404 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:35.404 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:35.404 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:35.404 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:35.404 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:35.404 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:35.404 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:35.404 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:35.404 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:35.405 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:35.405 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:35.405 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:35.405 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:35.405 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:35.405 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:35.405 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:35.405 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:35.405 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:35.405 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:35.405 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:35.405 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:35.405 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:35.406 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:35.406 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:35.406 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:35.406 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:35.406 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:35.406 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:35.406 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:35.406 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:35.678 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:35.679 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:35.679 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:35.679 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T11:26:35.679 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: failed to encode map e122 with expected crc 2024-03-24T11:26:35.679 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: failed to encode map e122 with expected crc 2024-03-24T11:26:35.679 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: osd.1 [v2:172.21.15.62:6810/3930446392,v1:172.21.15.62:6811/3930446392] boot 2024-03-24T11:26:35.679 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: osdmap e122: 8 total, 8 up, 8 in 2024-03-24T11:26:35.679 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T11:26:35.679 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: failed to encode map e122 with expected crc 2024-03-24T11:26:35.679 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: failed to encode map e122 with expected crc 2024-03-24T11:26:35.679 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: failed to encode map e122 with expected crc 2024-03-24T11:26:35.679 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: failed to encode map e122 with expected crc 2024-03-24T11:26:35.680 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: failed to encode map e122 with expected crc 2024-03-24T11:26:35.680 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:35.680 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:35.680 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:35.680 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:35.680 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:35.680 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:35.680 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:35.680 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:35.680 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:35.680 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:35.680 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:35.680 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:35.680 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:35.680 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:35.681 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:35.681 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:35.681 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:35.681 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:35.681 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:35.681 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:35.681 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:35.681 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:35.681 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:35.682 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:35.682 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:35.682 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:35.682 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:35.682 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:35.682 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:35.682 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:35.682 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:35.682 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:35.682 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:35.682 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:35.682 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:35.682 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:35.682 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:35.683 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:35.683 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:35.683 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:35.683 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:35.683 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:35.683 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:35.683 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:35.683 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:35.683 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:35.683 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:35.683 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:35.684 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:35.684 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:35.684 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:35.684 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T11:26:35.684 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: failed to encode map e122 with expected crc 2024-03-24T11:26:35.684 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: failed to encode map e122 with expected crc 2024-03-24T11:26:35.684 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: osd.1 [v2:172.21.15.62:6810/3930446392,v1:172.21.15.62:6811/3930446392] boot 2024-03-24T11:26:35.684 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: osdmap e122: 8 total, 8 up, 8 in 2024-03-24T11:26:35.684 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T11:26:35.684 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: failed to encode map e122 with expected crc 2024-03-24T11:26:35.684 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: failed to encode map e122 with expected crc 2024-03-24T11:26:35.685 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: failed to encode map e122 with expected crc 2024-03-24T11:26:35.685 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: failed to encode map e122 with expected crc 2024-03-24T11:26:35.685 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: failed to encode map e122 with expected crc 2024-03-24T11:26:35.685 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:35.685 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:35.685 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:35.685 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:35.685 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:35.685 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:35.685 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:35.685 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:35.685 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:35.685 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:35.685 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:35.685 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:35.686 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:35.686 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:35.686 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:35.686 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:35.686 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:35.686 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:35.686 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:35.686 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:35.686 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:35.686 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:35.686 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:35.687 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:35.687 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:35.687 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:35.687 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:35.687 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:35.687 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:35.687 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:35.687 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:35.687 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:35.687 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:35.688 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:35.688 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:35.688 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:35.688 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:35.688 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:35.688 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:35.688 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:35.688 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:35.688 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:35.688 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:35.688 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:35.688 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:35.688 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:35.688 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:35.688 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:35.689 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:35.689 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:35.689 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:35.689 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:35.689 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:35.689 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:35.689 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:35.689 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:35.689 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:35.689 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:35.689 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:35.689 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:35.689 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:35.689 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:35.689 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:35.690 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:36.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:36.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: pgmap v104: 161 pgs: 24 active+undersized+degraded, 39 active+undersized, 98 active+clean; 585 KiB data, 136 MiB used, 715 GiB / 715 GiB avail; 90/630 objects degraded (14.286%) 2024-03-24T11:26:36.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:36.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:36.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:36.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: failed to encode map e123 with expected crc 2024-03-24T11:26:36.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: failed to encode map e123 with expected crc 2024-03-24T11:26:36.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: osdmap e123: 8 total, 8 up, 8 in 2024-03-24T11:26:36.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: failed to encode map e123 with expected crc 2024-03-24T11:26:36.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: failed to encode map e123 with expected crc 2024-03-24T11:26:36.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: failed to encode map e123 with expected crc 2024-03-24T11:26:36.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: failed to encode map e123 with expected crc 2024-03-24T11:26:36.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: failed to encode map e123 with expected crc 2024-03-24T11:26:36.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: failed to encode map e123 with expected crc 2024-03-24T11:26:36.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: failed to encode map e123 with expected crc 2024-03-24T11:26:36.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: failed to encode map e123 with expected crc 2024-03-24T11:26:36.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: failed to encode map e123 with expected crc 2024-03-24T11:26:36.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:36.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:36.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:36.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:36.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:36.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:36.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:36.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:36.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:36.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:36.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:36.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:36.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:36.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:36.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:36.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:36.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:36.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:36.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:36.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:36.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:36.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:36.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:36.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:36.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:36.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:36.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:36.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:36.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:36.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:36.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:36.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:36.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:36.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:36.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:36.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:36.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:36.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:36.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:36.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:36.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:36.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:36.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:36.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:36.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:36.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:36.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:36.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:36.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:36.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:36.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:36.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:36.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:36.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:36.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:36.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:36.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:36.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:36.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: pgmap v104: 161 pgs: 24 active+undersized+degraded, 39 active+undersized, 98 active+clean; 585 KiB data, 136 MiB used, 715 GiB / 715 GiB avail; 90/630 objects degraded (14.286%) 2024-03-24T11:26:36.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:36.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:36.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:36.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: failed to encode map e123 with expected crc 2024-03-24T11:26:36.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: failed to encode map e123 with expected crc 2024-03-24T11:26:36.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: osdmap e123: 8 total, 8 up, 8 in 2024-03-24T11:26:36.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: failed to encode map e123 with expected crc 2024-03-24T11:26:36.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: failed to encode map e123 with expected crc 2024-03-24T11:26:36.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: failed to encode map e123 with expected crc 2024-03-24T11:26:36.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: failed to encode map e123 with expected crc 2024-03-24T11:26:36.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: failed to encode map e123 with expected crc 2024-03-24T11:26:36.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: failed to encode map e123 with expected crc 2024-03-24T11:26:36.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: failed to encode map e123 with expected crc 2024-03-24T11:26:36.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: failed to encode map e123 with expected crc 2024-03-24T11:26:36.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: failed to encode map e123 with expected crc 2024-03-24T11:26:36.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:36.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:36.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:36.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:36.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:36.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:36.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:36.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:36.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:36.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:36.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:36.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:36.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:36.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:36.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:36.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:36.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:36.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:36.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: pgmap v104: 161 pgs: 24 active+undersized+degraded, 39 active+undersized, 98 active+clean; 585 KiB data, 136 MiB used, 715 GiB / 715 GiB avail; 90/630 objects degraded (14.286%) 2024-03-24T11:26:36.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:36.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:36.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:36.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: failed to encode map e123 with expected crc 2024-03-24T11:26:36.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: failed to encode map e123 with expected crc 2024-03-24T11:26:36.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: osdmap e123: 8 total, 8 up, 8 in 2024-03-24T11:26:36.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: failed to encode map e123 with expected crc 2024-03-24T11:26:36.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: failed to encode map e123 with expected crc 2024-03-24T11:26:36.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: failed to encode map e123 with expected crc 2024-03-24T11:26:36.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: failed to encode map e123 with expected crc 2024-03-24T11:26:36.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: failed to encode map e123 with expected crc 2024-03-24T11:26:36.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: failed to encode map e123 with expected crc 2024-03-24T11:26:36.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: failed to encode map e123 with expected crc 2024-03-24T11:26:36.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: failed to encode map e123 with expected crc 2024-03-24T11:26:36.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: failed to encode map e123 with expected crc 2024-03-24T11:26:36.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:36.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:36.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:36.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:36.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:36.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:36.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:36.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:36.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:36.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:36.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:36.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:36.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:36.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:36.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:36.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:36.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:36.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:36.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:36.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:36.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:36.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:36.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:36.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:36.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:36.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:36.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:36.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:36.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:36.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:36.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:36.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:36.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:36.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:36.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:36.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:36.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:36.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:36.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:36.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:36.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:36.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:36.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:36.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:36.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:36.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:36.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:36.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:36.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:36.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:36.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:36.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:36.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:36.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:36.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:36.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:36.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:36.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:36.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:36.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:36.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:36.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:36.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:36.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:36.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:36.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:36.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:36.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:36.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:36.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:36.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:36.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:36.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:36.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:36.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:36.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:36.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:36.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:36.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:36.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:36.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:36.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:36.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:36.739 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:36.739 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:36.739 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:36.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:36.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:36.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:36.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:36.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:36.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:36.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:36.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:36.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:36.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:36.740 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:37.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:37.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:37.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:37.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:37.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:37.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:37.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:37.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:37.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:37.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:37.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:37.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:37.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:37.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:37.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:37.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:37.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:37.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:37.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:37.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:37.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:37.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:37.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:37.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:37.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:37.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:37.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:37.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:37.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:37.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:37.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:37.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:37.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:37.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:37.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:37.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:37.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:37.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:37.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:37.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:37.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:37.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:37.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:37.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:37.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:37.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:37.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:37.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:37.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:37.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:37.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:37.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:37.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:37.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:37.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:37.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:37.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:37.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:37.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:37.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:37.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:37.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:37.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:37.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:37.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:37.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:37.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:37.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:37.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:37.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:37.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:37.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:37.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:37.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:37.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:37.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:37.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:37.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:37.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:37.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:37.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:37.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:37.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:37.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:37.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:37.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:37.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:37 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:37.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:37 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:37.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:37 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:37.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:37.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:37.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:37.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:37.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:37.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:37 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:37.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:37.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:37.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:37.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:37.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:37.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:37 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:37.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:37.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:37.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:37.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:37.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:37.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:37 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:37.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:37.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:37.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:37.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:37.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:37.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:37 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:37.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:37.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:37.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:37.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:37.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:37.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:37 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:37.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:37.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:37.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:37.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:37.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:37.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:37.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:37.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:37.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:37.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:38.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:38 smithi139 ceph-mon[91894]: pgmap v106: 161 pgs: 9 active+undersized+degraded, 16 active+undersized, 136 active+clean; 585 KiB data, 136 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s; 41/630 objects degraded (6.508%) 2024-03-24T11:26:38.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:38 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:38.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:38 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:38.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:38.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:38.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:38.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:38.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:38.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:38 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:38.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:38.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:38.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:38.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:38.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:38.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:38 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:38.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:38.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:38.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:38.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:38.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:38.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:38 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:38.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:38.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:38.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:38.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:38.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:38.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:38.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:38.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-24T11:26:38.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:38 smithi139 ceph-mon[91894]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-24T11:26:38.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:38 smithi139 ceph-mon[91894]: Upgrade: osd.1 is safe to restart 2024-03-24T11:26:38.656 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[98643]: pgmap v106: 161 pgs: 9 active+undersized+degraded, 16 active+undersized, 136 active+clean; 585 KiB data, 136 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s; 41/630 objects degraded (6.508%) 2024-03-24T11:26:38.657 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:38.657 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:38.657 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:38.657 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:38.657 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:38.657 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:38.657 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:38.657 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:38.657 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:38.657 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:38.657 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:38.657 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:38.657 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:38.658 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:38.658 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:38.658 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:38.658 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:38.658 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:38.658 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:38.658 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:38.658 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:38.658 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:38.658 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:38.658 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:38.658 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:38.658 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:38.659 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:38.659 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-24T11:26:38.659 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[98643]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-24T11:26:38.659 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[98643]: Upgrade: osd.1 is safe to restart 2024-03-24T11:26:38.659 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[90553]: pgmap v106: 161 pgs: 9 active+undersized+degraded, 16 active+undersized, 136 active+clean; 585 KiB data, 136 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s; 41/630 objects degraded (6.508%) 2024-03-24T11:26:38.659 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:38.659 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:38.659 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:38.659 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:38.659 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:38.659 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:38.659 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:38.660 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:38.660 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:38.660 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:38.660 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:38.660 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:38.660 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:38.660 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:38.660 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:38.660 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:38.660 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:38.660 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:38.660 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:38.660 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:38.660 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:38.660 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:38.661 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:38.661 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:38.661 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:38.661 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:38.661 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:38.661 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-24T11:26:38.661 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[90553]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-24T11:26:38.661 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:38 smithi062 ceph-mon[90553]: Upgrade: osd.1 is safe to restart 2024-03-24T11:26:40.188 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:40 smithi062 ceph-mon[98643]: pgmap v107: 161 pgs: 161 active+clean; 585 KiB data, 136 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:26:40.189 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:40.189 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:40.189 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:40.189 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:40 smithi062 ceph-mon[98643]: Health check update: Degraded data redundancy: 41/630 objects degraded (6.508%), 9 pgs degraded (PG_DEGRADED) 2024-03-24T11:26:40.189 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:40 smithi062 ceph-mon[98643]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 41/630 objects degraded (6.508%), 9 pgs degraded) 2024-03-24T11:26:40.189 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:40 smithi062 ceph-mon[98643]: Cluster is now healthy 2024-03-24T11:26:40.189 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:40 smithi062 ceph-mon[90553]: pgmap v107: 161 pgs: 161 active+clean; 585 KiB data, 136 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:26:40.189 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:40.189 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:40.189 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:40.190 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:40 smithi062 ceph-mon[90553]: Health check update: Degraded data redundancy: 41/630 objects degraded (6.508%), 9 pgs degraded (PG_DEGRADED) 2024-03-24T11:26:40.190 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:40 smithi062 ceph-mon[90553]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 41/630 objects degraded (6.508%), 9 pgs degraded) 2024-03-24T11:26:40.190 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:40 smithi062 ceph-mon[90553]: Cluster is now healthy 2024-03-24T11:26:40.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:40 smithi139 ceph-mon[91894]: pgmap v107: 161 pgs: 161 active+clean; 585 KiB data, 136 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:26:40.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:40.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:40.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:40.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:40 smithi139 ceph-mon[91894]: Health check update: Degraded data redundancy: 41/630 objects degraded (6.508%), 9 pgs degraded (PG_DEGRADED) 2024-03-24T11:26:40.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:40 smithi139 ceph-mon[91894]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 41/630 objects degraded (6.508%), 9 pgs degraded) 2024-03-24T11:26:40.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:40 smithi139 ceph-mon[91894]: Cluster is now healthy 2024-03-24T11:26:40.977 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:26:40 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:26:40] "GET /metrics HTTP/1.1" 200 38492 "" "Prometheus/2.33.4" 2024-03-24T11:26:42.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:42 smithi062 ceph-mon[90553]: pgmap v108: 161 pgs: 161 active+clean; 585 KiB data, 136 MiB used, 715 GiB / 715 GiB avail; 658 B/s rd, 0 op/s 2024-03-24T11:26:42.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:42.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:26:42.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:42 smithi062 ceph-mon[90553]: Upgrade: Updating osd.1 2024-03-24T11:26:42.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-24T11:26:42.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:42.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:42 smithi062 ceph-mon[90553]: Deploying daemon osd.1 on smithi062 2024-03-24T11:26:42.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:42 smithi062 ceph-mon[98643]: pgmap v108: 161 pgs: 161 active+clean; 585 KiB data, 136 MiB used, 715 GiB / 715 GiB avail; 658 B/s rd, 0 op/s 2024-03-24T11:26:42.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:42.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:26:42.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:42 smithi062 ceph-mon[98643]: Upgrade: Updating osd.1 2024-03-24T11:26:42.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-24T11:26:42.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:42.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:42 smithi062 ceph-mon[98643]: Deploying daemon osd.1 on smithi062 2024-03-24T11:26:42.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:42 smithi139 ceph-mon[91894]: pgmap v108: 161 pgs: 161 active+clean; 585 KiB data, 136 MiB used, 715 GiB / 715 GiB avail; 658 B/s rd, 0 op/s 2024-03-24T11:26:42.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:42.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:26:42.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:42 smithi139 ceph-mon[91894]: Upgrade: Updating osd.1 2024-03-24T11:26:42.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-24T11:26:42.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:42.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:42 smithi139 ceph-mon[91894]: Deploying daemon osd.1 on smithi062 2024-03-24T11:26:43.956 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:43 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:26:43.956 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:43.956 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:43.956 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:43.956 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:43.956 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:43.957 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:43.957 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:43 smithi062 ceph-mon[98643]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:26:43.957 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:43.957 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:43.957 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:43.957 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:43.957 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:43.957 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:44.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:43 smithi139 ceph-mon[91894]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:26:44.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:44.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:44.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:44.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:44.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:44.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:44.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:44 smithi062 ceph-mon[90553]: pgmap v109: 161 pgs: 161 active+clean; 585 KiB data, 136 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-24T11:26:44.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:44 smithi062 ceph-mon[98643]: pgmap v109: 161 pgs: 161 active+clean; 585 KiB data, 136 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-24T11:26:45.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:44 smithi139 ceph-mon[91894]: pgmap v109: 161 pgs: 161 active+clean; 585 KiB data, 136 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-24T11:26:47.021 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:46 smithi062 ceph-mon[90553]: pgmap v110: 161 pgs: 161 active+clean; 585 KiB data, 136 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-24T11:26:47.021 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:47.021 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:46 smithi062 ceph-mon[98643]: pgmap v110: 161 pgs: 161 active+clean; 585 KiB data, 136 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-24T11:26:47.021 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:47.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:46 smithi139 ceph-mon[91894]: pgmap v110: 161 pgs: 161 active+clean; 585 KiB data, 136 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-24T11:26:47.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:48.246 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:48 smithi062 systemd[1]: Stopping Ceph osd.1 for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:26:48.726 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:48 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1[119928]: 2024-03-24T11:26:48.351+0000 7f7ff81c8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T11:26:48.727 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:48 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1[119928]: 2024-03-24T11:26:48.351+0000 7f7ff81c8700 -1 osd.1 123 *** Got signal Terminated *** 2024-03-24T11:26:48.727 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:48 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1[119928]: 2024-03-24T11:26:48.351+0000 7f7ff81c8700 -1 osd.1 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T11:26:49.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:48 smithi139 ceph-mon[91894]: pgmap v111: 161 pgs: 161 active+clean; 585 KiB data, 137 MiB used, 715 GiB / 715 GiB avail; 28 KiB/s rd, 0 B/s wr, 45 op/s 2024-03-24T11:26:49.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:48 smithi139 ceph-mon[91894]: osd.1 marked itself down and dead 2024-03-24T11:26:49.197 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:48 smithi062 ceph-mon[90553]: pgmap v111: 161 pgs: 161 active+clean; 585 KiB data, 137 MiB used, 715 GiB / 715 GiB avail; 28 KiB/s rd, 0 B/s wr, 45 op/s 2024-03-24T11:26:49.197 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:48 smithi062 ceph-mon[90553]: osd.1 marked itself down and dead 2024-03-24T11:26:49.197 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:48 smithi062 ceph-mon[98643]: pgmap v111: 161 pgs: 161 active+clean; 585 KiB data, 137 MiB used, 715 GiB / 715 GiB avail; 28 KiB/s rd, 0 B/s wr, 45 op/s 2024-03-24T11:26:49.197 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:48 smithi062 ceph-mon[98643]: osd.1 marked itself down and dead 2024-03-24T11:26:49.460 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:49 smithi062 podman[124437]: 2024-03-24 11:26:49.197221667 +0000 UTC m=+0.971691424 container died 51ec6bd391419265e5df614fc4db6829b253fbcfb5f9f70e90d68a6d3f01aa5e (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-db0330b, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS) 2024-03-24T11:26:49.726 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:49 smithi062 podman[124437]: 2024-03-24 11:26:49.65141211 +0000 UTC m=+1.425881875 container cleanup 51ec6bd391419265e5df614fc4db6829b253fbcfb5f9f70e90d68a6d3f01aa5e (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-db0330b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T11:26:49.727 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:49 smithi062 bash[124437]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1 2024-03-24T11:26:50.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:49 smithi139 ceph-mon[91894]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T11:26:50.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:49 smithi139 ceph-mon[91894]: failed to encode map e124 with expected crc 2024-03-24T11:26:50.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:49 smithi139 ceph-mon[91894]: osdmap e124: 8 total, 7 up, 8 in 2024-03-24T11:26:50.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:49 smithi139 ceph-mon[91894]: failed to encode map e124 with expected crc 2024-03-24T11:26:50.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:49 smithi139 ceph-mon[91894]: failed to encode map e124 with expected crc 2024-03-24T11:26:50.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:49 smithi139 ceph-mon[91894]: failed to encode map e124 with expected crc 2024-03-24T11:26:50.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:49 smithi139 ceph-mon[91894]: failed to encode map e124 with expected crc 2024-03-24T11:26:50.227 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:49 smithi062 podman[124456]: 2024-03-24 11:26:49.953688476 +0000 UTC m=+0.755684361 container remove 51ec6bd391419265e5df614fc4db6829b253fbcfb5f9f70e90d68a6d3f01aa5e (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-db0330b, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T11:26:50.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:49 smithi062 ceph-mon[90553]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T11:26:50.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:49 smithi062 ceph-mon[90553]: failed to encode map e124 with expected crc 2024-03-24T11:26:50.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:49 smithi062 ceph-mon[90553]: osdmap e124: 8 total, 7 up, 8 in 2024-03-24T11:26:50.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:49 smithi062 ceph-mon[90553]: failed to encode map e124 with expected crc 2024-03-24T11:26:50.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:49 smithi062 ceph-mon[90553]: failed to encode map e124 with expected crc 2024-03-24T11:26:50.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:49 smithi062 ceph-mon[90553]: failed to encode map e124 with expected crc 2024-03-24T11:26:50.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:49 smithi062 ceph-mon[90553]: failed to encode map e124 with expected crc 2024-03-24T11:26:50.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:49 smithi062 ceph-mon[98643]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T11:26:50.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:49 smithi062 ceph-mon[98643]: failed to encode map e124 with expected crc 2024-03-24T11:26:50.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:49 smithi062 ceph-mon[98643]: osdmap e124: 8 total, 7 up, 8 in 2024-03-24T11:26:50.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:49 smithi062 ceph-mon[98643]: failed to encode map e124 with expected crc 2024-03-24T11:26:50.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:49 smithi062 ceph-mon[98643]: failed to encode map e124 with expected crc 2024-03-24T11:26:50.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:49 smithi062 ceph-mon[98643]: failed to encode map e124 with expected crc 2024-03-24T11:26:50.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:49 smithi062 ceph-mon[98643]: failed to encode map e124 with expected crc 2024-03-24T11:26:50.557 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:50 smithi062 podman[124546]: 2024-03-24 11:26:50.188571784 +0000 UTC m=+0.018467158 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:26:50.558 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:50 smithi062 podman[124546]: 2024-03-24 11:26:50.313399079 +0000 UTC m=+0.143294448 container create c01fb32cc64ba0b1c00a3a98e62aafebccbe9eac18add5b2be69d28b6dd65015 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-db0330b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T11:26:50.862 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:26:50 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:26:50] "GET /metrics HTTP/1.1" 200 38500 "" "Prometheus/2.33.4" 2024-03-24T11:26:50.862 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:50 smithi062 podman[124546]: 2024-03-24 11:26:50.557862997 +0000 UTC m=+0.387758385 container init c01fb32cc64ba0b1c00a3a98e62aafebccbe9eac18add5b2be69d28b6dd65015 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-db0330b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-24T11:26:50.862 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:50 smithi062 podman[124546]: 2024-03-24 11:26:50.561180809 +0000 UTC m=+0.391076188 container start c01fb32cc64ba0b1c00a3a98e62aafebccbe9eac18add5b2be69d28b6dd65015 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1-deactivate, GIT_CLEAN=True, ceph=True, RELEASE=squid-db0330b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-24T11:26:50.862 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:50 smithi062 podman[124546]: 2024-03-24 11:26:50.63263891 +0000 UTC m=+0.462534279 container attach c01fb32cc64ba0b1c00a3a98e62aafebccbe9eac18add5b2be69d28b6dd65015 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1-deactivate, RELEASE=squid-db0330b, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T11:26:50.863 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:50 smithi062 ceph-mon[90553]: pgmap v113: 161 pgs: 6 peering, 19 stale+active+clean, 136 active+clean; 585 KiB data, 137 MiB used, 715 GiB / 715 GiB avail; 44 KiB/s rd, 0 B/s wr, 73 op/s 2024-03-24T11:26:50.863 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:50 smithi062 ceph-mon[90553]: failed to encode map e125 with expected crc 2024-03-24T11:26:50.863 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:50 smithi062 ceph-mon[90553]: failed to encode map e125 with expected crc 2024-03-24T11:26:50.863 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:50 smithi062 ceph-mon[90553]: osdmap e125: 8 total, 7 up, 8 in 2024-03-24T11:26:50.863 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:50 smithi062 ceph-mon[90553]: failed to encode map e125 with expected crc 2024-03-24T11:26:50.863 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:50 smithi062 ceph-mon[90553]: failed to encode map e125 with expected crc 2024-03-24T11:26:50.863 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:50 smithi062 ceph-mon[90553]: failed to encode map e125 with expected crc 2024-03-24T11:26:50.863 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:50 smithi062 ceph-mon[90553]: failed to encode map e125 with expected crc 2024-03-24T11:26:50.863 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:50 smithi062 ceph-mon[90553]: failed to encode map e125 with expected crc 2024-03-24T11:26:50.863 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:50 smithi062 ceph-mon[90553]: failed to encode map e125 with expected crc 2024-03-24T11:26:50.863 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:50 smithi062 ceph-mon[90553]: failed to encode map e125 with expected crc 2024-03-24T11:26:50.864 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:50 smithi062 ceph-mon[90553]: failed to encode map e125 with expected crc 2024-03-24T11:26:50.864 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:50 smithi062 ceph-mon[98643]: pgmap v113: 161 pgs: 6 peering, 19 stale+active+clean, 136 active+clean; 585 KiB data, 137 MiB used, 715 GiB / 715 GiB avail; 44 KiB/s rd, 0 B/s wr, 73 op/s 2024-03-24T11:26:50.864 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:50 smithi062 ceph-mon[98643]: failed to encode map e125 with expected crc 2024-03-24T11:26:50.864 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:50 smithi062 ceph-mon[98643]: failed to encode map e125 with expected crc 2024-03-24T11:26:50.864 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:50 smithi062 ceph-mon[98643]: osdmap e125: 8 total, 7 up, 8 in 2024-03-24T11:26:50.864 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:50 smithi062 ceph-mon[98643]: failed to encode map e125 with expected crc 2024-03-24T11:26:50.864 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:50 smithi062 ceph-mon[98643]: failed to encode map e125 with expected crc 2024-03-24T11:26:50.864 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:50 smithi062 ceph-mon[98643]: failed to encode map e125 with expected crc 2024-03-24T11:26:50.864 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:50 smithi062 ceph-mon[98643]: failed to encode map e125 with expected crc 2024-03-24T11:26:50.864 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:50 smithi062 ceph-mon[98643]: failed to encode map e125 with expected crc 2024-03-24T11:26:50.864 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:50 smithi062 ceph-mon[98643]: failed to encode map e125 with expected crc 2024-03-24T11:26:50.864 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:50 smithi062 ceph-mon[98643]: failed to encode map e125 with expected crc 2024-03-24T11:26:50.864 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:50 smithi062 ceph-mon[98643]: failed to encode map e125 with expected crc 2024-03-24T11:26:51.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:50 smithi139 ceph-mon[91894]: pgmap v113: 161 pgs: 6 peering, 19 stale+active+clean, 136 active+clean; 585 KiB data, 137 MiB used, 715 GiB / 715 GiB avail; 44 KiB/s rd, 0 B/s wr, 73 op/s 2024-03-24T11:26:51.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:50 smithi139 ceph-mon[91894]: failed to encode map e125 with expected crc 2024-03-24T11:26:51.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:50 smithi139 ceph-mon[91894]: failed to encode map e125 with expected crc 2024-03-24T11:26:51.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:50 smithi139 ceph-mon[91894]: osdmap e125: 8 total, 7 up, 8 in 2024-03-24T11:26:51.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:50 smithi139 ceph-mon[91894]: failed to encode map e125 with expected crc 2024-03-24T11:26:51.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:50 smithi139 ceph-mon[91894]: failed to encode map e125 with expected crc 2024-03-24T11:26:51.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:50 smithi139 ceph-mon[91894]: failed to encode map e125 with expected crc 2024-03-24T11:26:51.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:50 smithi139 ceph-mon[91894]: failed to encode map e125 with expected crc 2024-03-24T11:26:51.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:50 smithi139 ceph-mon[91894]: failed to encode map e125 with expected crc 2024-03-24T11:26:51.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:50 smithi139 ceph-mon[91894]: failed to encode map e125 with expected crc 2024-03-24T11:26:51.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:50 smithi139 ceph-mon[91894]: failed to encode map e125 with expected crc 2024-03-24T11:26:51.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:50 smithi139 ceph-mon[91894]: failed to encode map e125 with expected crc 2024-03-24T11:26:51.203 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:50 smithi062 podman[124546]: 2024-03-24 11:26:50.863088669 +0000 UTC m=+0.692984054 container died c01fb32cc64ba0b1c00a3a98e62aafebccbe9eac18add5b2be69d28b6dd65015 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=squid-db0330b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-24T11:26:51.977 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:51 smithi062 podman[124573]: 2024-03-24 11:26:51.629391907 +0000 UTC m=+0.757637945 container remove c01fb32cc64ba0b1c00a3a98e62aafebccbe9eac18add5b2be69d28b6dd65015 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-24T11:26:51.977 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:51 smithi062 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.1.service: Deactivated successfully. 2024-03-24T11:26:51.977 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:51 smithi062 systemd[1]: Stopped Ceph osd.1 for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:26:51.977 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:51 smithi062 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.1.service: Consumed 1.936s CPU time. 2024-03-24T11:26:51.977 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:51 smithi062 systemd[1]: Starting Ceph osd.1 for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:26:52.341 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:52 smithi062 podman[124681]: 2024-03-24 11:26:51.979837 +0000 UTC m=+0.017823378 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:26:52.341 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:52 smithi062 podman[124681]: 2024-03-24 11:26:52.113799744 +0000 UTC m=+0.151786105 container create 1f1d3f7406d2bf2902322056e80fb57782f807593e705edf08ada84dadc072d9 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1-activate, CEPH_POINT_RELEASE=, RELEASE=squid-db0330b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307) 2024-03-24T11:26:52.342 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:52 smithi062 podman[124681]: 2024-03-24 11:26:52.341603346 +0000 UTC m=+0.379589723 container init 1f1d3f7406d2bf2902322056e80fb57782f807593e705edf08ada84dadc072d9 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1-activate, RELEASE=squid-db0330b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T11:26:52.726 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:52 smithi062 podman[124681]: 2024-03-24 11:26:52.345460739 +0000 UTC m=+0.383447099 container start 1f1d3f7406d2bf2902322056e80fb57782f807593e705edf08ada84dadc072d9 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1-activate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, RELEASE=squid-db0330b) 2024-03-24T11:26:52.727 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:52 smithi062 podman[124681]: 2024-03-24 11:26:52.441383748 +0000 UTC m=+0.479370117 container attach 1f1d3f7406d2bf2902322056e80fb57782f807593e705edf08ada84dadc072d9 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1-activate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-db0330b, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS) 2024-03-24T11:26:53.086 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:52 smithi062 ceph-mon[90553]: pgmap v115: 161 pgs: 6 peering, 19 stale+active+clean, 136 active+clean; 585 KiB data, 137 MiB used, 715 GiB / 715 GiB avail; 55 KiB/s rd, 0 B/s wr, 91 op/s 2024-03-24T11:26:53.086 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:52 smithi062 ceph-mon[98643]: pgmap v115: 161 pgs: 6 peering, 19 stale+active+clean, 136 active+clean; 585 KiB data, 137 MiB used, 715 GiB / 715 GiB avail; 55 KiB/s rd, 0 B/s wr, 91 op/s 2024-03-24T11:26:53.086 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:52 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1-activate[124696]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-24T11:26:53.086 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:52 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1-activate[124696]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-24T11:26:53.086 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:52 smithi062 bash[124681]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-24T11:26:53.087 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:52 smithi062 bash[124681]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-24T11:26:53.087 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:52 smithi062 bash[124681]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-24T11:26:53.087 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:52 smithi062 bash[124681]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-24T11:26:53.087 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:52 smithi062 bash[124681]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-03-24T11:26:53.087 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:52 smithi062 bash[124681]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-24T11:26:53.087 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:52 smithi062 bash[124681]: --> ceph-volume raw activate successful for osd ID: 1 2024-03-24T11:26:53.087 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:52 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1-activate[124696]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-24T11:26:53.087 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:52 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1-activate[124696]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-24T11:26:53.087 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:52 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1-activate[124696]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-03-24T11:26:53.087 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:52 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1-activate[124696]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-24T11:26:53.087 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:52 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1-activate[124696]: --> ceph-volume raw activate successful for osd ID: 1 2024-03-24T11:26:53.087 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:52 smithi062 podman[124681]: 2024-03-24 11:26:52.814689344 +0000 UTC m=+0.852675704 container died 1f1d3f7406d2bf2902322056e80fb57782f807593e705edf08ada84dadc072d9 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1-activate, maintainer=Guillaume Abrioux , RELEASE=squid-db0330b, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-24T11:26:53.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:52 smithi139 ceph-mon[91894]: pgmap v115: 161 pgs: 6 peering, 19 stale+active+clean, 136 active+clean; 585 KiB data, 137 MiB used, 715 GiB / 715 GiB avail; 55 KiB/s rd, 0 B/s wr, 91 op/s 2024-03-24T11:26:53.722 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:53 smithi062 podman[124827]: 2024-03-24 11:26:53.454407947 +0000 UTC m=+0.634680806 container remove 1f1d3f7406d2bf2902322056e80fb57782f807593e705edf08ada84dadc072d9 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1-activate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-db0330b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T11:26:53.722 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:53 smithi062 podman[124893]: 2024-03-24 11:26:53.569619142 +0000 UTC m=+0.017830821 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:26:53.722 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:53 smithi062 podman[124893]: 2024-03-24 11:26:53.722020085 +0000 UTC m=+0.170231750 container create c443b3fe97c1affcb852f369d64fd050cfad2baa18a8dfbc554a5b3785d9949a (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-db0330b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T11:26:53.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:53 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:26:53.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:53 smithi062 ceph-mon[98643]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:26:53.977 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:53 smithi062 podman[124893]: 2024-03-24 11:26:53.966370002 +0000 UTC m=+0.414581665 container init c443b3fe97c1affcb852f369d64fd050cfad2baa18a8dfbc554a5b3785d9949a (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, ceph=True, RELEASE=squid-db0330b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T11:26:53.977 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:53 smithi062 podman[124893]: 2024-03-24 11:26:53.970491039 +0000 UTC m=+0.418702704 container start c443b3fe97c1affcb852f369d64fd050cfad2baa18a8dfbc554a5b3785d9949a (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-db0330b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T11:26:54.080 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:53 smithi139 ceph-mon[91894]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:26:54.294 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:53 smithi062 ceph-osd[124913]: -- 172.21.15.62:0/3230648678 <== mon.0 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56178f4d11e0 con 0x56178f4a3400 2024-03-24T11:26:54.294 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:54 smithi062 bash[124893]: c443b3fe97c1affcb852f369d64fd050cfad2baa18a8dfbc554a5b3785d9949a 2024-03-24T11:26:54.294 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:54 smithi062 systemd[1]: Started Ceph osd.1 for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:26:54.624 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1[124909]: 2024-03-24T11:26:54.511+0000 7f5f19e7c7c0 -1 Falling back to public interface 2024-03-24T11:26:54.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[90553]: pgmap v116: 161 pgs: 22 active+undersized+degraded, 6 peering, 1 stale+active+clean, 34 active+undersized, 98 active+clean; 585 KiB data, 143 MiB used, 715 GiB / 715 GiB avail; 98 KiB/s rd, 0 B/s wr, 163 op/s; 83/630 objects degraded (13.175%) 2024-03-24T11:26:54.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[90553]: Health check failed: Degraded data redundancy: 83/630 objects degraded (13.175%), 22 pgs degraded (PG_DEGRADED) 2024-03-24T11:26:54.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:54.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:54.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:54.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:54.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:54.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:54.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:54.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:54.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:54.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:54.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:54.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:54.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:54.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:54.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:54.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:54.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:54.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:54.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:54.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:54.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:54.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:54.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:54.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:54.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:54.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[98643]: pgmap v116: 161 pgs: 22 active+undersized+degraded, 6 peering, 1 stale+active+clean, 34 active+undersized, 98 active+clean; 585 KiB data, 143 MiB used, 715 GiB / 715 GiB avail; 98 KiB/s rd, 0 B/s wr, 163 op/s; 83/630 objects degraded (13.175%) 2024-03-24T11:26:54.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[98643]: Health check failed: Degraded data redundancy: 83/630 objects degraded (13.175%), 22 pgs degraded (PG_DEGRADED) 2024-03-24T11:26:54.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:54.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:54.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:54.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:54.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:54.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:54.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:54.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:54.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:54.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:54.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:54.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:54.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:54.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:54.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:54.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:54.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:54.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:54.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:54.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:54.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:54.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:54.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:54.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:54.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:54.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:54.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:54.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:54.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:54.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:54.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:54.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:54.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:54.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:54.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:54.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:54.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:54.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:54.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:54.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:54.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:54.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:54.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:54.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:54.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:54 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:55.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:54 smithi139 ceph-mon[91894]: pgmap v116: 161 pgs: 22 active+undersized+degraded, 6 peering, 1 stale+active+clean, 34 active+undersized, 98 active+clean; 585 KiB data, 143 MiB used, 715 GiB / 715 GiB avail; 98 KiB/s rd, 0 B/s wr, 163 op/s; 83/630 objects degraded (13.175%) 2024-03-24T11:26:55.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:54 smithi139 ceph-mon[91894]: Health check failed: Degraded data redundancy: 83/630 objects degraded (13.175%), 22 pgs degraded (PG_DEGRADED) 2024-03-24T11:26:55.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:54 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:55.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:54 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:55.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:54 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:55.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:54 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:55.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:54 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:55.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:54 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:55.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:54 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:55.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:54 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:55.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:54 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:55.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:54 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:55.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:54 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:55.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:54 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:55.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:54 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:55.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:54 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:55.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:54 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:55.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:54 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:55.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:54 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:55.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:54 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:55.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:54 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:55.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:54 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:55.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:54 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:55.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:54 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:55.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:54 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:55.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:54 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:55.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:54 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:55.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:54 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:55.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:54 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:55.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:54 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:55.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:54 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:55.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:54 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:55.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:54 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:55.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:54 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:55.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:54 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:55.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:54 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:55.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:54 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:55.911 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:55.911 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:55.911 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:55.912 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:55.912 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:55.912 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:55.912 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:55.912 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:55.912 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:55.912 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:55.912 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:55.912 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:55.912 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:55.912 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:55.912 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:55.912 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:55.913 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:55.913 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:55.913 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:55.913 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:55.913 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:55.913 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:55.913 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:55.913 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:55.913 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:55.913 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:55.913 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:55.913 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:55.914 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:55.914 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:55.914 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:55.914 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:55.914 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:55.914 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:55.914 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:55.915 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:55.915 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:55.915 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:55.915 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:55.915 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:55.915 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:55.915 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:55.915 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:55.915 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:55.915 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:55.916 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:55.916 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:55.916 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:55.916 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:55.916 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:55.916 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:55.916 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:55.916 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:55.916 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:55.916 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:55.916 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:55.916 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:55.916 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:55.917 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:55.917 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:55.917 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:55.917 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:55.917 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:55.917 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:55.917 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:55.917 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:55.917 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:55.917 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:55.917 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:55.917 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:55.917 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:55.918 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:55.918 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:55.918 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:55.918 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:55.918 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:55.918 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:55.918 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:55.918 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:55.919 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:55.919 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:55.919 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:55.919 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:55.919 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:55.919 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:55.919 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:55.919 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:55.920 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:55.920 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:55.920 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:55.920 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:55.920 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:55.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:55.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:55.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:55.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:55.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:55.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:55.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:55.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:55.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:55.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:55.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:55.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:55.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:55.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:55.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:55.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:55.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:55.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:55.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:55.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:55.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:55.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:55.922 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:55.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:55.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:55.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:55.923 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:55.923 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:56.060 INFO:teuthology.orchestra.run.smithi062.stdout:true 2024-03-24T11:26:56.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:56.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:56.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:56.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:56.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:56.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:56.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:56.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:56.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:56.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:56.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:56.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:56.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:56.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:56.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:56.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:56.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:56.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:56.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:56.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:56.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:56.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:56.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:56.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:56.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:56.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:56.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:56.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:56.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:56.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:56.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:56.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:56.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:56.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:56.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:56.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:56.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:56.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:56.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:56.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:56.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:56.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:56.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:56.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:56.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:56.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:56.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:56.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:56.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:56.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:56.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:56.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:56.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:56.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:56.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:56.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:56.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:56.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:56.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:56.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:56.815 INFO:teuthology.orchestra.run.smithi062.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T11:26:56.816 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi062 smithi062 running 13s ago 16m - - 2024-03-24T11:26:56.816 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi139 smithi139 running 17s ago 16m - - 2024-03-24T11:26:56.816 INFO:teuthology.orchestra.run.smithi062.stdout:alertmanager.a smithi062 *:9093,9094 running (6m) 13s ago 10m 24.4M - 0.23.0 ba2b418f427c 818298d58140 2024-03-24T11:26:56.816 INFO:teuthology.orchestra.run.smithi062.stdout:grafana.a smithi139 *:3000 running (5m) 17s ago 10m 53.8M - 8.3.5 dad864ee21e9 b48167e7171e 2024-03-24T11:26:56.816 INFO:teuthology.orchestra.run.smithi062.stdout:iscsi.foo.smithi062.dggsvk smithi062 running (4m) 13s ago 9m 57.1M - 3.5 e1d6a67b021e 0a98806ec839 2024-03-24T11:26:56.816 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.x smithi139 *:8443,9283,8765 running (3m) 17s ago 15m 450M - 19.0.0-1667-gdb0330b1 4f325f8440e9 d94987b11387 2024-03-24T11:26:56.816 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.y smithi062 *:8443,9283,8765 running (5m) 13s ago 17m 519M - 19.0.0-1667-gdb0330b1 4f325f8440e9 ca1a49028d65 2024-03-24T11:26:56.816 INFO:teuthology.orchestra.run.smithi062.stdout:mon.a smithi062 running (2m) 13s ago 17m 53.8M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 f7afe6e28c88 2024-03-24T11:26:56.816 INFO:teuthology.orchestra.run.smithi062.stdout:mon.b smithi139 running (2m) 17s ago 15m 37.9M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 ca357beea4dd 2024-03-24T11:26:56.816 INFO:teuthology.orchestra.run.smithi062.stdout:mon.c smithi062 running (2m) 13s ago 16m 46.8M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 459e7449ec39 2024-03-24T11:26:56.816 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.a smithi062 *:9100 running (6m) 13s ago 11m 20.9M - 1.3.1 1dbe0e931976 79d5a02a999f 2024-03-24T11:26:56.816 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.b smithi139 *:9100 running (5m) 17s ago 11m 15.7M - 1.3.1 1dbe0e931976 8f9f4608e465 2024-03-24T11:26:56.816 INFO:teuthology.orchestra.run.smithi062.stdout:osd.0 smithi062 running (52s) 13s ago 15m 49.6M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 dfe4920b49c4 2024-03-24T11:26:56.816 INFO:teuthology.orchestra.run.smithi062.stdout:osd.1 smithi062 starting - - - 4096M 2024-03-24T11:26:56.816 INFO:teuthology.orchestra.run.smithi062.stdout:osd.2 smithi062 running (14m) 13s ago 14m 53.1M 4096M 17.2.0 e1d6a67b021e 714f550103a7 2024-03-24T11:26:56.817 INFO:teuthology.orchestra.run.smithi062.stdout:osd.3 smithi062 running (14m) 13s ago 14m 57.6M 4096M 17.2.0 e1d6a67b021e 1d2868bd5195 2024-03-24T11:26:56.817 INFO:teuthology.orchestra.run.smithi062.stdout:osd.4 smithi139 running (13m) 17s ago 13m 60.0M 4096M 17.2.0 e1d6a67b021e 7568bfa9e74d 2024-03-24T11:26:56.817 INFO:teuthology.orchestra.run.smithi062.stdout:osd.5 smithi139 running (13m) 17s ago 13m 54.7M 4096M 17.2.0 e1d6a67b021e dd0abd599a14 2024-03-24T11:26:56.817 INFO:teuthology.orchestra.run.smithi062.stdout:osd.6 smithi139 running (13m) 17s ago 13m 54.2M 4096M 17.2.0 e1d6a67b021e a4b1289aca69 2024-03-24T11:26:56.817 INFO:teuthology.orchestra.run.smithi062.stdout:osd.7 smithi139 running (12m) 17s ago 12m 56.5M 4096M 17.2.0 e1d6a67b021e bc7fadbc1384 2024-03-24T11:26:56.817 INFO:teuthology.orchestra.run.smithi062.stdout:prometheus.a smithi139 *:9095 running (4m) 17s ago 11m 71.1M - 2.33.4 514e6a882f6e 2ed47f810d33 2024-03-24T11:26:56.817 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi062.sacpds smithi062 *:8000 running (10m) 13s ago 10m 73.6M - 17.2.0 e1d6a67b021e 89f4ac308e98 2024-03-24T11:26:56.817 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi139.yljesh smithi139 *:8000 running (10m) 17s ago 10m 72.5M - 17.2.0 e1d6a67b021e e189bdc12cae 2024-03-24T11:26:56.817 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi062.xvpxnk smithi062 *:80 running (9m) 13s ago 9m 73.5M - 17.2.0 e1d6a67b021e d1aa74adab44 2024-03-24T11:26:56.817 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi139.pbtrmp smithi139 *:80 running (9m) 17s ago 9m 76.1M - 17.2.0 e1d6a67b021e 7f8e336755d7 2024-03-24T11:26:57.027 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:57.027 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: pgmap v117: 161 pgs: 24 active+undersized+degraded, 39 active+undersized, 98 active+clean; 585 KiB data, 143 MiB used, 715 GiB / 715 GiB avail; 68 KiB/s rd, 0 B/s wr, 113 op/s; 90/630 objects degraded (14.286%) 2024-03-24T11:26:57.027 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:57.027 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:57.027 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:57.027 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:57.027 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:57.027 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:57.027 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:57.027 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:57.027 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:57.027 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:57.027 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:57.027 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:57.028 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:57.028 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:57.028 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:57.028 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:57.028 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:57.028 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:57.028 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:57.028 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:57.028 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:57.028 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:57.028 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:57.028 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:57.028 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:26:57.028 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:57.028 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:57.029 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:57.029 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: pgmap v117: 161 pgs: 24 active+undersized+degraded, 39 active+undersized, 98 active+clean; 585 KiB data, 143 MiB used, 715 GiB / 715 GiB avail; 68 KiB/s rd, 0 B/s wr, 113 op/s; 90/630 objects degraded (14.286%) 2024-03-24T11:26:57.029 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:57.029 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:57.029 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:57.029 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:57.029 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:57.029 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:57.029 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:57.029 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:57.029 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:57.030 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:57.030 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:57.030 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:57.030 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:57.030 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:57.030 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:57.030 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:57.030 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:57.030 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:57.030 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:57.030 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:57.030 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:57.030 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:57.030 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:57.030 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:57.031 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:26:57.031 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:57.031 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:57.031 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:57.031 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:57.031 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:57.031 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:57.031 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:57.031 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:57.031 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:57.032 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:57.032 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:57.032 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:57.032 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:57.032 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:57.032 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:57.032 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:57.032 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:57.032 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:57.032 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:57.032 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:57.032 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:57.032 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:57.033 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:57.033 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:57.033 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:57.033 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:57.033 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:57.033 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:57.033 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:57.033 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:57.033 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:57.033 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:57.033 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:57.033 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:57.033 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:57.033 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:57.034 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:57.034 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:57.034 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:57.034 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:57.034 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:57.034 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:57.034 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:57.034 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:57.034 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:57.034 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:57.034 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:57.034 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:57.034 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:57.034 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:57.035 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:57.035 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:57.035 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:57.035 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:57.035 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:57.035 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:57.105 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:57.105 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: pgmap v117: 161 pgs: 24 active+undersized+degraded, 39 active+undersized, 98 active+clean; 585 KiB data, 143 MiB used, 715 GiB / 715 GiB avail; 68 KiB/s rd, 0 B/s wr, 113 op/s; 90/630 objects degraded (14.286%) 2024-03-24T11:26:57.106 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:57.106 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:57.106 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:57.106 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:57.106 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:57.106 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:57.106 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:57.106 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:57.106 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:57.106 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:57.106 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:57.106 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:57.106 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:57.106 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:57.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:57.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:57.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:57.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:57.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:57.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:57.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:57.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:57.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:57.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:57.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:26:57.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:57.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:57.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:57.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:57.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:57.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:57.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:57.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:57.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:57.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:57.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:57.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:57.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:57.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:57.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:57.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:57.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:57.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:57.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:57.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:57.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:57.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:57.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:57.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:57.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:57.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:57.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:57.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:57.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:57.320 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1[124909]: 2024-03-24T11:26:57.262+0000 7f5f19e7c7c0 -1 osd.1 123 log_to_monitors true 2024-03-24T11:26:57.320 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:26:57.320 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": { 2024-03-24T11:26:57.320 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 3 2024-03-24T11:26:57.320 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:26:57.320 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": { 2024-03-24T11:26:57.320 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-24T11:26:57.320 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:26:57.320 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": { 2024-03-24T11:26:57.320 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 6, 2024-03-24T11:26:57.320 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 1 2024-03-24T11:26:57.321 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:26:57.321 INFO:teuthology.orchestra.run.smithi062.stdout: "rgw": { 2024-03-24T11:26:57.321 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4 2024-03-24T11:26:57.321 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:26:57.321 INFO:teuthology.orchestra.run.smithi062.stdout: "overall": { 2024-03-24T11:26:57.321 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 10, 2024-03-24T11:26:57.321 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 6 2024-03-24T11:26:57.321 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-24T11:26:57.321 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:26:57.686 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:26:57.686 INFO:teuthology.orchestra.run.smithi062.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69", 2024-03-24T11:26:57.686 INFO:teuthology.orchestra.run.smithi062.stdout: "in_progress": true, 2024-03-24T11:26:57.686 INFO:teuthology.orchestra.run.smithi062.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T11:26:57.686 INFO:teuthology.orchestra.run.smithi062.stdout: "services_complete": [ 2024-03-24T11:26:57.686 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr", 2024-03-24T11:26:57.686 INFO:teuthology.orchestra.run.smithi062.stdout: "mon" 2024-03-24T11:26:57.686 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-03-24T11:26:57.686 INFO:teuthology.orchestra.run.smithi062.stdout: "progress": "6/23 daemons upgraded", 2024-03-24T11:26:57.686 INFO:teuthology.orchestra.run.smithi062.stdout: "message": "Currently upgrading osd daemons", 2024-03-24T11:26:57.686 INFO:teuthology.orchestra.run.smithi062.stdout: "is_paused": false 2024-03-24T11:26:57.687 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:26:57.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:57.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: from='client.64267 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:26:57.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:57.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:57.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:57.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: from='client.64273 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:26:57.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:57.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:57.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:57.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:57.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: from='client.44313 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:26:57.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:57.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:57.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:57.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:57.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:57.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:57.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:57.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:57.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:57.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:57.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:57.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:57.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:57.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:57.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:57.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:57.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:57.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: from='osd.1 [v2:172.21.15.62:6810/1563233908,v1:172.21.15.62:6811/1563233908]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-24T11:26:57.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:57.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: from='client.64267 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:26:57.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:57.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:57.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:57.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: from='client.64273 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:26:57.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:57.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:57.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:57.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:57.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: from='client.44313 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:26:57.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:57.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:57.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:57.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:57.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:57.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:57.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:57.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:57.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:57.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:57.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:57.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:57.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:57.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:57.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:57.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:57.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:57.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: from='osd.1 [v2:172.21.15.62:6810/1563233908,v1:172.21.15.62:6811/1563233908]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-24T11:26:57.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:57.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:57.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:57.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:57.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:57.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: from='client.? 172.21.15.62:0/3693053490' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:57.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:57.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:57.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:57.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:57.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:57.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:57.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:57.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:57.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:57.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:57.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:57.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:57.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:57.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:57.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:57.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:57.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:57.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:57.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:57.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:57.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:57.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:57.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:57.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:57.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:57.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: from='client.? 172.21.15.62:0/3693053490' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:57.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:57.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:57.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:57.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:57.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:57.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:57.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:57.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:57.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:57.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:57.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:57.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:57.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:57.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:57.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:57.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:57.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:57.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:57.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:57.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:58.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:58.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: from='client.64267 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:26:58.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:58.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:58.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:58.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: from='client.64273 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:26:58.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:58.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:58.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:58.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:58.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: from='client.44313 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:26:58.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:58.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:58.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:58.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:58.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:58.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:58.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:58.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:58.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:58.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:58.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:58.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:58.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:58.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:58.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:58.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:58.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:58.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: from='osd.1 [v2:172.21.15.62:6810/1563233908,v1:172.21.15.62:6811/1563233908]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-24T11:26:58.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:58.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:58.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:58.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:58.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:58.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: from='client.? 172.21.15.62:0/3693053490' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:58.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:58.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:58.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:58.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:58.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:58.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:58.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:58.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:58.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:58.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:58.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:58.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:58.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:58.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:58.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:58.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:58.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:58.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:58.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:58.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:58.152 INFO:teuthology.orchestra.run.smithi062.stdout:HEALTH_WARN 1 osds down; Degraded data redundancy: 90/630 objects degraded (14.286%), 24 pgs degraded 2024-03-24T11:26:58.153 INFO:teuthology.orchestra.run.smithi062.stdout:[WRN] OSD_DOWN: 1 osds down 2024-03-24T11:26:58.153 INFO:teuthology.orchestra.run.smithi062.stdout: osd.1 (root=default,host=smithi062) is down 2024-03-24T11:26:58.153 INFO:teuthology.orchestra.run.smithi062.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 90/630 objects degraded (14.286%), 24 pgs degraded 2024-03-24T11:26:58.153 INFO:teuthology.orchestra.run.smithi062.stdout: pg 2.0 is active+undersized+degraded, acting [7,0] 2024-03-24T11:26:58.153 INFO:teuthology.orchestra.run.smithi062.stdout: pg 2.2 is active+undersized+degraded, acting [5,6] 2024-03-24T11:26:58.153 INFO:teuthology.orchestra.run.smithi062.stdout: pg 2.4 is active+undersized+degraded, acting [0,7] 2024-03-24T11:26:58.153 INFO:teuthology.orchestra.run.smithi062.stdout: pg 2.9 is active+undersized+degraded, acting [7,3] 2024-03-24T11:26:58.153 INFO:teuthology.orchestra.run.smithi062.stdout: pg 2.a is active+undersized+degraded, acting [3,7] 2024-03-24T11:26:58.153 INFO:teuthology.orchestra.run.smithi062.stdout: pg 2.15 is active+undersized+degraded, acting [5,0] 2024-03-24T11:26:58.153 INFO:teuthology.orchestra.run.smithi062.stdout: pg 3.0 is active+undersized+degraded, acting [2,6] 2024-03-24T11:26:58.153 INFO:teuthology.orchestra.run.smithi062.stdout: pg 3.4 is active+undersized+degraded, acting [2,5] 2024-03-24T11:26:58.153 INFO:teuthology.orchestra.run.smithi062.stdout: pg 3.6 is active+undersized+degraded, acting [0,4] 2024-03-24T11:26:58.153 INFO:teuthology.orchestra.run.smithi062.stdout: pg 3.8 is active+undersized+degraded, acting [3,7] 2024-03-24T11:26:58.153 INFO:teuthology.orchestra.run.smithi062.stdout: pg 3.a is active+undersized+degraded, acting [6,4] 2024-03-24T11:26:58.154 INFO:teuthology.orchestra.run.smithi062.stdout: pg 3.e is active+undersized+degraded, acting [7,4] 2024-03-24T11:26:58.154 INFO:teuthology.orchestra.run.smithi062.stdout: pg 3.16 is active+undersized+degraded, acting [5,7] 2024-03-24T11:26:58.154 INFO:teuthology.orchestra.run.smithi062.stdout: pg 3.18 is active+undersized+degraded, acting [3,0] 2024-03-24T11:26:58.154 INFO:teuthology.orchestra.run.smithi062.stdout: pg 3.19 is active+undersized+degraded, acting [3,4] 2024-03-24T11:26:58.154 INFO:teuthology.orchestra.run.smithi062.stdout: pg 3.1a is active+undersized+degraded, acting [4,2] 2024-03-24T11:26:58.154 INFO:teuthology.orchestra.run.smithi062.stdout: pg 3.1c is active+undersized+degraded, acting [5,4] 2024-03-24T11:26:58.154 INFO:teuthology.orchestra.run.smithi062.stdout: pg 4.9 is active+undersized+degraded, acting [4,3] 2024-03-24T11:26:58.154 INFO:teuthology.orchestra.run.smithi062.stdout: pg 4.d is active+undersized+degraded, acting [4,2] 2024-03-24T11:26:58.154 INFO:teuthology.orchestra.run.smithi062.stdout: pg 4.e is active+undersized+degraded, acting [4,6] 2024-03-24T11:26:58.154 INFO:teuthology.orchestra.run.smithi062.stdout: pg 4.14 is active+undersized+degraded, acting [3,7] 2024-03-24T11:26:58.154 INFO:teuthology.orchestra.run.smithi062.stdout: pg 4.1f is active+undersized+degraded, acting [6,5] 2024-03-24T11:26:58.154 INFO:teuthology.orchestra.run.smithi062.stdout: pg 5.1d is active+undersized+degraded, acting [4,0] 2024-03-24T11:26:58.154 INFO:teuthology.orchestra.run.smithi062.stdout: pg 6.1a is active+undersized+degraded, acting [4,5] 2024-03-24T11:26:58.407 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1[124909]: 2024-03-24T11:26:58.163+0000 7f5f0b2d4700 -1 osd.1 123 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T11:26:58.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: pgmap v118: 161 pgs: 24 active+undersized+degraded, 39 active+undersized, 98 active+clean; 585 KiB data, 143 MiB used, 715 GiB / 715 GiB avail; 59 KiB/s rd, 0 B/s wr, 98 op/s; 90/630 objects degraded (14.286%) 2024-03-24T11:26:58.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:58.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:58.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:58.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:58.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:58.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: from='client.44328 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:26:58.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:58.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:58.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:58.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:58.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:58.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:58.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:58.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:58.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:58.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:58.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:58.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:58.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:58.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:58.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:58.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:58.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:58.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:58.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: from='osd.1 [v2:172.21.15.62:6810/1563233908,v1:172.21.15.62:6811/1563233908]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-24T11:26:58.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: osdmap e126: 8 total, 7 up, 8 in 2024-03-24T11:26:58.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: from='osd.1 [v2:172.21.15.62:6810/1563233908,v1:172.21.15.62:6811/1563233908]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:26:58.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: from='client.? 172.21.15.62:0/1595798535' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:26:58.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:58.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:58.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:58.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:58.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:58.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:58.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:58.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:58.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:58.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:58.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:58.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:58.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:58.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:58.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:58.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:58.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:58.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:58.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:58.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:58.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:58.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:58.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:58.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:58.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:58.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: pgmap v118: 161 pgs: 24 active+undersized+degraded, 39 active+undersized, 98 active+clean; 585 KiB data, 143 MiB used, 715 GiB / 715 GiB avail; 59 KiB/s rd, 0 B/s wr, 98 op/s; 90/630 objects degraded (14.286%) 2024-03-24T11:26:58.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:58.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:58.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:58.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:58.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:58.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: from='client.44328 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:26:58.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:58.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:58.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:58.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:58.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:58.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:58.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:58.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:58.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:58.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:58.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:58.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:58.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:58.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:58.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:58.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:58.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:58.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:58.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: from='osd.1 [v2:172.21.15.62:6810/1563233908,v1:172.21.15.62:6811/1563233908]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-24T11:26:58.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: osdmap e126: 8 total, 7 up, 8 in 2024-03-24T11:26:58.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: from='osd.1 [v2:172.21.15.62:6810/1563233908,v1:172.21.15.62:6811/1563233908]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:26:58.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: from='client.? 172.21.15.62:0/1595798535' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:26:58.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:58.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:58.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:58.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:58.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:58.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:58.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:58.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:58.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:58.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:58.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:58.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:58.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:58.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:58.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:58.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:58.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:58.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:58.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:58.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:58.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:58.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:58.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:58.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:58.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:59.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: pgmap v118: 161 pgs: 24 active+undersized+degraded, 39 active+undersized, 98 active+clean; 585 KiB data, 143 MiB used, 715 GiB / 715 GiB avail; 59 KiB/s rd, 0 B/s wr, 98 op/s; 90/630 objects degraded (14.286%) 2024-03-24T11:26:59.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:59.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:59.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:59.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:59.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:59.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: from='client.44328 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:26:59.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:59.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:59.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:59.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:59.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:59.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:59.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:59.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:59.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:59.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:59.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:59.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:59.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:26:59.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:59.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:59.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:59.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:59.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:59.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: from='osd.1 [v2:172.21.15.62:6810/1563233908,v1:172.21.15.62:6811/1563233908]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-24T11:26:59.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: osdmap e126: 8 total, 7 up, 8 in 2024-03-24T11:26:59.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: from='osd.1 [v2:172.21.15.62:6810/1563233908,v1:172.21.15.62:6811/1563233908]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:26:59.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: from='client.? 172.21.15.62:0/1595798535' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:26:59.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:59.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:59.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:59.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:59.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:59.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:59.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:59.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:59.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:59.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:59.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:59.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:59.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:59.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:59.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:59.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:59.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:59.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:59.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:59.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:26:59.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:26:59.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:26:59.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:26:59.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:26:59.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:00.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: failed to encode map e126 with expected crc 2024-03-24T11:27:00.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: failed to encode map e126 with expected crc 2024-03-24T11:27:00.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:00.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:00.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:00.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:00.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:00.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:00.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: failed to encode map e126 with expected crc 2024-03-24T11:27:00.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: failed to encode map e126 with expected crc 2024-03-24T11:27:00.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:00.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T11:27:00.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: osd.1 [v2:172.21.15.62:6810/1563233908,v1:172.21.15.62:6811/1563233908] boot 2024-03-24T11:27:00.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: osdmap e127: 8 total, 8 up, 8 in 2024-03-24T11:27:00.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T11:27:00.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: failed to encode map e127 with expected crc 2024-03-24T11:27:00.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: failed to encode map e127 with expected crc 2024-03-24T11:27:00.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: Health check update: Degraded data redundancy: 90/630 objects degraded (14.286%), 24 pgs degraded (PG_DEGRADED) 2024-03-24T11:27:00.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:26:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:00.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: failed to encode map e126 with expected crc 2024-03-24T11:27:00.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: failed to encode map e126 with expected crc 2024-03-24T11:27:00.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:00.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:00.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:00.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:00.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:00.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:00.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: failed to encode map e126 with expected crc 2024-03-24T11:27:00.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: failed to encode map e126 with expected crc 2024-03-24T11:27:00.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:00.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:00.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: failed to encode map e126 with expected crc 2024-03-24T11:27:00.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: failed to encode map e126 with expected crc 2024-03-24T11:27:00.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:00.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:00.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:00.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:00.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:00.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:00.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: failed to encode map e126 with expected crc 2024-03-24T11:27:00.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: failed to encode map e126 with expected crc 2024-03-24T11:27:00.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:00.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T11:27:00.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: osd.1 [v2:172.21.15.62:6810/1563233908,v1:172.21.15.62:6811/1563233908] boot 2024-03-24T11:27:00.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: osdmap e127: 8 total, 8 up, 8 in 2024-03-24T11:27:00.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T11:27:00.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: failed to encode map e127 with expected crc 2024-03-24T11:27:00.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: failed to encode map e127 with expected crc 2024-03-24T11:27:00.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: Health check update: Degraded data redundancy: 90/630 objects degraded (14.286%), 24 pgs degraded (PG_DEGRADED) 2024-03-24T11:27:00.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T11:27:00.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: osd.1 [v2:172.21.15.62:6810/1563233908,v1:172.21.15.62:6811/1563233908] boot 2024-03-24T11:27:00.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: osdmap e127: 8 total, 8 up, 8 in 2024-03-24T11:27:00.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T11:27:00.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: failed to encode map e127 with expected crc 2024-03-24T11:27:00.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: failed to encode map e127 with expected crc 2024-03-24T11:27:00.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: Health check update: Degraded data redundancy: 90/630 objects degraded (14.286%), 24 pgs degraded (PG_DEGRADED) 2024-03-24T11:27:00.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:26:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.956 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:27:00] "GET /metrics HTTP/1.1" 200 38511 "" "Prometheus/2.33.4" 2024-03-24T11:27:00.956 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:00.957 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: pgmap v120: 161 pgs: 24 active+undersized+degraded, 39 active+undersized, 98 active+clean; 585 KiB data, 143 MiB used, 715 GiB / 715 GiB avail; 46 KiB/s rd, 0 B/s wr, 76 op/s; 90/630 objects degraded (14.286%) 2024-03-24T11:27:00.957 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:00.957 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: failed to encode map e127 with expected crc 2024-03-24T11:27:00.957 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: failed to encode map e127 with expected crc 2024-03-24T11:27:00.957 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: failed to encode map e127 with expected crc 2024-03-24T11:27:00.957 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: failed to encode map e127 with expected crc 2024-03-24T11:27:00.957 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: failed to encode map e127 with expected crc 2024-03-24T11:27:00.957 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:00.957 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:00.957 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:00.957 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:00.957 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:00.958 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:00.958 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.958 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.958 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.958 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.958 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.958 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:00.958 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.958 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.958 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.958 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.958 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:00.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:00.959 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:00.960 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: pgmap v120: 161 pgs: 24 active+undersized+degraded, 39 active+undersized, 98 active+clean; 585 KiB data, 143 MiB used, 715 GiB / 715 GiB avail; 46 KiB/s rd, 0 B/s wr, 76 op/s; 90/630 objects degraded (14.286%) 2024-03-24T11:27:00.960 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:00.960 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: failed to encode map e127 with expected crc 2024-03-24T11:27:00.960 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: failed to encode map e127 with expected crc 2024-03-24T11:27:00.960 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: failed to encode map e127 with expected crc 2024-03-24T11:27:00.960 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: failed to encode map e127 with expected crc 2024-03-24T11:27:00.960 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: failed to encode map e127 with expected crc 2024-03-24T11:27:00.960 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:00.960 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:00.960 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:00.960 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:00.960 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:00.960 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:00.961 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.961 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.961 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.961 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.961 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.961 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:00.961 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.961 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.961 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.961 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.961 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.961 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:00.962 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.962 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.962 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.962 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.962 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.962 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:00.962 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.963 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.963 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.963 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.963 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.963 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.963 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.963 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: failed to encode map e128 with expected crc 2024-03-24T11:27:00.963 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: osdmap e128: 8 total, 8 up, 8 in 2024-03-24T11:27:00.963 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.963 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.963 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: failed to encode map e128 with expected crc 2024-03-24T11:27:00.963 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: failed to encode map e128 with expected crc 2024-03-24T11:27:00.964 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: failed to encode map e128 with expected crc 2024-03-24T11:27:00.964 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: failed to encode map e128 with expected crc 2024-03-24T11:27:00.964 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.964 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.964 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.964 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.964 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.964 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.964 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.964 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.964 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.964 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.967 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.967 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.967 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.967 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.967 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.967 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.967 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.967 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: failed to encode map e128 with expected crc 2024-03-24T11:27:00.967 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: osdmap e128: 8 total, 8 up, 8 in 2024-03-24T11:27:00.967 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.967 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.968 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: failed to encode map e128 with expected crc 2024-03-24T11:27:00.968 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: failed to encode map e128 with expected crc 2024-03-24T11:27:00.968 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: failed to encode map e128 with expected crc 2024-03-24T11:27:00.968 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: failed to encode map e128 with expected crc 2024-03-24T11:27:00.968 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.968 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.968 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.968 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.968 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.968 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.969 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.969 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.969 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.969 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.969 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.969 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.969 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.969 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.969 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.969 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.969 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.969 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.970 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.970 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.970 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.970 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.970 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.970 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.970 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.970 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:00.970 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:00.970 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:00.970 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:00.970 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:00.970 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:01.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:01.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: pgmap v120: 161 pgs: 24 active+undersized+degraded, 39 active+undersized, 98 active+clean; 585 KiB data, 143 MiB used, 715 GiB / 715 GiB avail; 46 KiB/s rd, 0 B/s wr, 76 op/s; 90/630 objects degraded (14.286%) 2024-03-24T11:27:01.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:01.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: failed to encode map e127 with expected crc 2024-03-24T11:27:01.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: failed to encode map e127 with expected crc 2024-03-24T11:27:01.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: failed to encode map e127 with expected crc 2024-03-24T11:27:01.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: failed to encode map e127 with expected crc 2024-03-24T11:27:01.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: failed to encode map e127 with expected crc 2024-03-24T11:27:01.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:01.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:01.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:01.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:01.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:01.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:01.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:01.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:01.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:01.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:01.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:01.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:01.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:01.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:01.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:01.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:01.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:01.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:01.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:01.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:01.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:01.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:01.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:01.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:01.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:01.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:01.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:01.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:01.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:01.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:01.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:01.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: failed to encode map e128 with expected crc 2024-03-24T11:27:01.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: osdmap e128: 8 total, 8 up, 8 in 2024-03-24T11:27:01.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:01.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:01.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: failed to encode map e128 with expected crc 2024-03-24T11:27:01.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: failed to encode map e128 with expected crc 2024-03-24T11:27:01.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: failed to encode map e128 with expected crc 2024-03-24T11:27:01.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: failed to encode map e128 with expected crc 2024-03-24T11:27:01.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:01.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:01.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:01.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:01.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:01.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:01.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:01.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:01.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:01.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:01.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:01.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:01.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:01.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:01.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:01.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:01.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:01.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:01.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:01.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:01.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:01.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:01.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:01.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:01.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:01.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:01.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:01.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:01.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:01.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:01.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:02.032 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:02.032 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: failed to encode map e128 with expected crc 2024-03-24T11:27:02.032 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: failed to encode map e128 with expected crc 2024-03-24T11:27:02.032 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: failed to encode map e128 with expected crc 2024-03-24T11:27:02.032 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: failed to encode map e128 with expected crc 2024-03-24T11:27:02.032 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: failed to encode map e128 with expected crc 2024-03-24T11:27:02.032 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: failed to encode map e128 with expected crc 2024-03-24T11:27:02.032 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: failed to encode map e128 with expected crc 2024-03-24T11:27:02.033 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: failed to encode map e128 with expected crc 2024-03-24T11:27:02.033 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: failed to encode map e128 with expected crc 2024-03-24T11:27:02.033 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: failed to encode map e128 with expected crc 2024-03-24T11:27:02.033 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: failed to encode map e128 with expected crc 2024-03-24T11:27:02.033 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:02.033 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:02.033 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:02.033 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:02.033 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:02.033 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:02.033 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:02.033 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:02.033 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:02.033 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:02.033 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:02.034 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:02.034 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:02.034 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:02.034 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:02.034 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:02.034 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:02.034 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:02.034 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:02.034 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:02.034 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:02.034 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:02.035 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:02.035 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: failed to encode map e128 with expected crc 2024-03-24T11:27:02.035 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: failed to encode map e128 with expected crc 2024-03-24T11:27:02.035 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: failed to encode map e128 with expected crc 2024-03-24T11:27:02.035 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: failed to encode map e128 with expected crc 2024-03-24T11:27:02.035 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: failed to encode map e128 with expected crc 2024-03-24T11:27:02.035 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: failed to encode map e128 with expected crc 2024-03-24T11:27:02.035 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: failed to encode map e128 with expected crc 2024-03-24T11:27:02.035 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: failed to encode map e128 with expected crc 2024-03-24T11:27:02.035 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: failed to encode map e128 with expected crc 2024-03-24T11:27:02.035 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: failed to encode map e128 with expected crc 2024-03-24T11:27:02.035 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: failed to encode map e128 with expected crc 2024-03-24T11:27:02.035 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:02.036 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:02.036 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:02.036 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:02.036 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:02.036 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:02.036 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:02.036 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:02.036 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:02.036 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:02.036 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:02.036 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:02.036 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:02.036 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:02.036 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:02.037 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:02.037 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:02.037 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:02.037 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:02.037 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:02.037 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:02.037 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:02.037 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:02.037 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:02.037 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:02.037 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:02.038 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:02.038 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:02.038 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:02.038 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:02.038 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:02.038 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:02.038 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:02.038 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:02.038 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:02.038 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:02.038 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:02.038 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:02.038 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:02.038 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:02.039 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:02.039 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:02.039 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:02.039 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:02.039 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:02.039 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:02.039 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:02.039 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:02.039 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:02.039 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:02.039 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:02.039 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:02.039 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:02.039 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:02.039 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:02.040 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:02.040 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:02.040 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:02.040 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:02.040 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:02.040 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:02.040 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:02.040 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:02.040 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:02.040 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:02.041 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:02.041 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:02.041 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:02.041 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:02.041 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:02.041 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:02.041 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:02.041 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:02.041 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:02.041 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:02.041 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:02.041 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:02.041 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:02.041 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:02.042 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:02.042 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:02.042 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:02.042 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:02.042 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:02.042 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:02.042 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:02.042 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:02.042 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:02.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:02.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: failed to encode map e128 with expected crc 2024-03-24T11:27:02.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: failed to encode map e128 with expected crc 2024-03-24T11:27:02.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: failed to encode map e128 with expected crc 2024-03-24T11:27:02.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: failed to encode map e128 with expected crc 2024-03-24T11:27:02.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: failed to encode map e128 with expected crc 2024-03-24T11:27:02.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: failed to encode map e128 with expected crc 2024-03-24T11:27:02.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: failed to encode map e128 with expected crc 2024-03-24T11:27:02.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: failed to encode map e128 with expected crc 2024-03-24T11:27:02.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: failed to encode map e128 with expected crc 2024-03-24T11:27:02.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: failed to encode map e128 with expected crc 2024-03-24T11:27:02.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: failed to encode map e128 with expected crc 2024-03-24T11:27:02.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:02.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:02.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:02.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:02.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:02.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:02.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:02.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:02.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:02.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:02.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:02.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:02.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:02.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:02.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:02.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:02.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:02.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:02.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:02.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:02.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:02.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:02.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:02.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:02.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:02.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:02.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:02.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:02.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:02.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:02.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:02.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:02.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:02.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:02.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:02.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:02.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:02.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:02.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:02.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:02.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:02.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:02.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:02.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:02.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:02.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:02.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:02.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:02.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:02.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:02.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:02.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:02.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:02.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:02.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:02.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:02.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: pgmap v123: 161 pgs: 24 active+undersized+degraded, 39 active+undersized, 98 active+clean; 585 KiB data, 143 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 90/630 objects degraded (14.286%) 2024-03-24T11:27:02.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:02.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:02.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:02.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:02.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:02.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:02.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:02.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:02.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:02.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:02.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:02.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:02.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:02.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:02.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:02.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:02.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:02.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:02.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:02.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:02.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:02.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:02.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:02.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:02.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:02.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:02.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:02.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:02.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: pgmap v123: 161 pgs: 24 active+undersized+degraded, 39 active+undersized, 98 active+clean; 585 KiB data, 143 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 90/630 objects degraded (14.286%) 2024-03-24T11:27:02.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:02.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:02.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:02.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:02.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:02.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:02.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:02.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:02.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:02.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:02.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:02.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:02.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:02.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:02.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:02.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:02.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:02.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:02.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:02.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:02.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:02.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:02.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:02.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:02.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:02.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:02.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:02.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:02.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:02.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:02.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:02.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:02.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:02.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:02.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:02.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:02.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:02.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:02.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:02.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:02.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:02.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:02.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:02.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:02.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:02.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:02.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:02.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:02.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:02.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:02.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:02.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:02.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:02.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:02.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:02.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:02.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:02.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:02.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:02.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:02.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:02.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:02.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:02.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:02.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:02.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:02.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:02.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:02.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:02.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:02.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:02.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:02.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:02.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:02.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:02.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:02.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:02.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:02.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:03.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:03.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: pgmap v123: 161 pgs: 24 active+undersized+degraded, 39 active+undersized, 98 active+clean; 585 KiB data, 143 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 90/630 objects degraded (14.286%) 2024-03-24T11:27:03.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:03.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:03.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:03.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:03.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:03.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:03.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:03.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:03.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:03.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:03.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:03.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:03.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:03.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:03.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:03.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:03.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:03.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:03.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:03.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:03.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:03.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:03.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:03.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:03.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:03.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:03.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:03.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:03.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:03.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:03.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:03.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:03.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:03.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:03.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:03.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:03.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:03.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:03.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:03.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:03.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:03.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:03.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:03.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:03.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:03.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:03.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:03.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:03.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:03.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:03.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:03.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:03.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:03.726 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:27:03.427+0000 7f4e4aad7700 -1 mgr.server reply reply (16) Device or resource busy unsafe to stop osd(s) at this time (6 PGs are or would become offline) 2024-03-24T11:27:04.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:03 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:04.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:03 smithi139 ceph-mon[91894]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:27:04.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:03 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:04.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:03 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:04.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:03 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:04.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:03 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:04.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:03 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:04.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:03 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:04.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:04.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:04.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:04.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:04.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:04.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:03 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:04.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:04.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:04.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:04.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:04.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:04.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:03 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:04.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:04.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:04.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:04.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:04.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:04.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:03 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:04.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:04.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:04.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:04.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:04.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:04.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:04.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:04.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:04.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:04.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:04.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:04.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:04.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:04.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:04.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:04.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:04.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:04.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:04.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:04.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:04.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:04.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:04.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-24T11:27:04.123 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:04.123 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[98643]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:27:04.123 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:04.123 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:04.123 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:04.124 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:04.124 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:04.124 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:04.124 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:04.124 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:04.124 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:04.124 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:04.124 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:04.124 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:04.124 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:04.124 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:04.124 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:04.124 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:04.125 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:04.125 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:04.125 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:04.125 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:04.125 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:04.125 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:04.125 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:04.125 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:04.125 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:04.125 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:04.125 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:04.125 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:04.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:04.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:27:04.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:04.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:04.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:04.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:04.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:04.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:04.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:04.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:04.126 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:04.127 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:04.127 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:04.127 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:04.127 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:04.127 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:04.127 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:04.127 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:04.127 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:04.127 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:04.127 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:04.127 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:04.127 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:04.127 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:04.128 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:04.128 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:04.128 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:04.128 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:04.128 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:04.128 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:04.128 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:04.128 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:04.128 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:04.128 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:04.128 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:04.129 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:04.129 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:04.129 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:04.129 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:04.129 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:04.129 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:04.129 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:04.129 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:04.129 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:04.129 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:04.129 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:04.129 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:04.129 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:04.129 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-24T11:27:04.130 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:04.130 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:04.130 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:04.130 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:04.130 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:04.130 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:04.130 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:04.130 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:04.130 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:04.130 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:04.130 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:04.130 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:04.131 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:04.131 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:04.131 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:04.131 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:04.131 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:04.131 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:04.131 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-24T11:27:05.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:05 smithi062 ceph-mon[90553]: pgmap v124: 161 pgs: 9 active+undersized+degraded, 16 active+undersized, 136 active+clean; 585 KiB data, 144 MiB used, 715 GiB / 715 GiB avail; 27 KiB/s rd, 0 B/s wr, 44 op/s; 41/630 objects degraded (6.508%); 0 B/s, 5 objects/s recovering 2024-03-24T11:27:05.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:05 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:05.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:05 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:05.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:05 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:05.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:05 smithi062 ceph-mon[90553]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-24T11:27:05.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:05 smithi062 ceph-mon[90553]: Upgrade: unsafe to stop osd(s) at this time (6 PGs are or would become offline) 2024-03-24T11:27:05.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:05 smithi062 ceph-mon[90553]: Health check update: Degraded data redundancy: 41/630 objects degraded (6.508%), 9 pgs degraded (PG_DEGRADED) 2024-03-24T11:27:05.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:05 smithi062 ceph-mon[98643]: pgmap v124: 161 pgs: 9 active+undersized+degraded, 16 active+undersized, 136 active+clean; 585 KiB data, 144 MiB used, 715 GiB / 715 GiB avail; 27 KiB/s rd, 0 B/s wr, 44 op/s; 41/630 objects degraded (6.508%); 0 B/s, 5 objects/s recovering 2024-03-24T11:27:05.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:05 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:05.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:05 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:05.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:05 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:05.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:05 smithi062 ceph-mon[98643]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-24T11:27:05.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:05 smithi062 ceph-mon[98643]: Upgrade: unsafe to stop osd(s) at this time (6 PGs are or would become offline) 2024-03-24T11:27:05.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:05 smithi062 ceph-mon[98643]: Health check update: Degraded data redundancy: 41/630 objects degraded (6.508%), 9 pgs degraded (PG_DEGRADED) 2024-03-24T11:27:05.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:05 smithi139 ceph-mon[91894]: pgmap v124: 161 pgs: 9 active+undersized+degraded, 16 active+undersized, 136 active+clean; 585 KiB data, 144 MiB used, 715 GiB / 715 GiB avail; 27 KiB/s rd, 0 B/s wr, 44 op/s; 41/630 objects degraded (6.508%); 0 B/s, 5 objects/s recovering 2024-03-24T11:27:05.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:05 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:05.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:05 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:05.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:05 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:05.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:05 smithi139 ceph-mon[91894]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-24T11:27:05.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:05 smithi139 ceph-mon[91894]: Upgrade: unsafe to stop osd(s) at this time (6 PGs are or would become offline) 2024-03-24T11:27:05.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:05 smithi139 ceph-mon[91894]: Health check update: Degraded data redundancy: 41/630 objects degraded (6.508%), 9 pgs degraded (PG_DEGRADED) 2024-03-24T11:27:06.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:06 smithi062 ceph-mon[90553]: pgmap v125: 161 pgs: 161 active+clean; 585 KiB data, 144 MiB used, 715 GiB / 715 GiB avail; 41 KiB/s rd, 0 B/s wr, 68 op/s; 0 B/s, 8 objects/s recovering 2024-03-24T11:27:06.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:06 smithi062 ceph-mon[90553]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 41/630 objects degraded (6.508%), 9 pgs degraded) 2024-03-24T11:27:06.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:06 smithi062 ceph-mon[90553]: Cluster is now healthy 2024-03-24T11:27:06.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:06 smithi062 ceph-mon[98643]: pgmap v125: 161 pgs: 161 active+clean; 585 KiB data, 144 MiB used, 715 GiB / 715 GiB avail; 41 KiB/s rd, 0 B/s wr, 68 op/s; 0 B/s, 8 objects/s recovering 2024-03-24T11:27:06.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:06 smithi062 ceph-mon[98643]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 41/630 objects degraded (6.508%), 9 pgs degraded) 2024-03-24T11:27:06.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:06 smithi062 ceph-mon[98643]: Cluster is now healthy 2024-03-24T11:27:06.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:06 smithi139 ceph-mon[91894]: pgmap v125: 161 pgs: 161 active+clean; 585 KiB data, 144 MiB used, 715 GiB / 715 GiB avail; 41 KiB/s rd, 0 B/s wr, 68 op/s; 0 B/s, 8 objects/s recovering 2024-03-24T11:27:06.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:06 smithi139 ceph-mon[91894]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 41/630 objects degraded (6.508%), 9 pgs degraded) 2024-03-24T11:27:06.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:06 smithi139 ceph-mon[91894]: Cluster is now healthy 2024-03-24T11:27:07.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:07.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:07.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:07.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:07.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:07.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:07.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:07.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:07.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:07.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:07.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:07.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:08.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:08.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:08.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:08.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:08.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:08.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:08.846 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:08 smithi062 ceph-mon[90553]: pgmap v126: 161 pgs: 161 active+clean; 585 KiB data, 153 MiB used, 715 GiB / 715 GiB avail; 93 KiB/s rd, 0 B/s wr, 153 op/s; 0 B/s, 7 objects/s recovering 2024-03-24T11:27:08.847 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:08 smithi062 ceph-mon[98643]: pgmap v126: 161 pgs: 161 active+clean; 585 KiB data, 153 MiB used, 715 GiB / 715 GiB avail; 93 KiB/s rd, 0 B/s wr, 153 op/s; 0 B/s, 7 objects/s recovering 2024-03-24T11:27:09.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:08 smithi139 ceph-mon[91894]: pgmap v126: 161 pgs: 161 active+clean; 585 KiB data, 153 MiB used, 715 GiB / 715 GiB avail; 93 KiB/s rd, 0 B/s wr, 153 op/s; 0 B/s, 7 objects/s recovering 2024-03-24T11:27:11.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:10 smithi139 ceph-mon[91894]: pgmap v127: 161 pgs: 161 active+clean; 585 KiB data, 153 MiB used, 715 GiB / 715 GiB avail; 87 KiB/s rd, 0 B/s wr, 144 op/s; 0 B/s, 5 objects/s recovering 2024-03-24T11:27:11.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:27:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:27:10] "GET /metrics HTTP/1.1" 200 38511 "" "Prometheus/2.33.4" 2024-03-24T11:27:11.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:10 smithi062 ceph-mon[90553]: pgmap v127: 161 pgs: 161 active+clean; 585 KiB data, 153 MiB used, 715 GiB / 715 GiB avail; 87 KiB/s rd, 0 B/s wr, 144 op/s; 0 B/s, 5 objects/s recovering 2024-03-24T11:27:11.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:10 smithi062 ceph-mon[98643]: pgmap v127: 161 pgs: 161 active+clean; 585 KiB data, 153 MiB used, 715 GiB / 715 GiB avail; 87 KiB/s rd, 0 B/s wr, 144 op/s; 0 B/s, 5 objects/s recovering 2024-03-24T11:27:12.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:12 smithi062 ceph-mon[90553]: pgmap v128: 161 pgs: 161 active+clean; 585 KiB data, 153 MiB used, 715 GiB / 715 GiB avail; 79 KiB/s rd, 0 B/s wr, 131 op/s; 0 B/s, 5 objects/s recovering 2024-03-24T11:27:12.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:12.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:27:12.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:12.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:12 smithi062 ceph-mon[98643]: pgmap v128: 161 pgs: 161 active+clean; 585 KiB data, 153 MiB used, 715 GiB / 715 GiB avail; 79 KiB/s rd, 0 B/s wr, 131 op/s; 0 B/s, 5 objects/s recovering 2024-03-24T11:27:12.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:12.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:27:12.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:12.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:12 smithi139 ceph-mon[91894]: pgmap v128: 161 pgs: 161 active+clean; 585 KiB data, 153 MiB used, 715 GiB / 715 GiB avail; 79 KiB/s rd, 0 B/s wr, 131 op/s; 0 B/s, 5 objects/s recovering 2024-03-24T11:27:12.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:12.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:27:12.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:13.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:13 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:27:13.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:13 smithi062 ceph-mon[98643]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:27:13.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:13 smithi139 ceph-mon[91894]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:27:14.257 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:14 smithi139 ceph-mon[91894]: pgmap v129: 161 pgs: 161 active+clean; 585 KiB data, 153 MiB used, 715 GiB / 715 GiB avail; 73 KiB/s rd, 0 B/s wr, 120 op/s; 0 B/s, 4 objects/s recovering 2024-03-24T11:27:14.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:14 smithi062 ceph-mon[90553]: pgmap v129: 161 pgs: 161 active+clean; 585 KiB data, 153 MiB used, 715 GiB / 715 GiB avail; 73 KiB/s rd, 0 B/s wr, 120 op/s; 0 B/s, 4 objects/s recovering 2024-03-24T11:27:14.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:14 smithi062 ceph-mon[98643]: pgmap v129: 161 pgs: 161 active+clean; 585 KiB data, 153 MiB used, 715 GiB / 715 GiB avail; 73 KiB/s rd, 0 B/s wr, 120 op/s; 0 B/s, 4 objects/s recovering 2024-03-24T11:27:16.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:16 smithi062 ceph-mon[90553]: pgmap v130: 161 pgs: 161 active+clean; 585 KiB data, 153 MiB used, 715 GiB / 715 GiB avail; 59 KiB/s rd, 0 B/s wr, 98 op/s; 0 B/s, 1 objects/s recovering 2024-03-24T11:27:16.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:16 smithi062 ceph-mon[98643]: pgmap v130: 161 pgs: 161 active+clean; 585 KiB data, 153 MiB used, 715 GiB / 715 GiB avail; 59 KiB/s rd, 0 B/s wr, 98 op/s; 0 B/s, 1 objects/s recovering 2024-03-24T11:27:16.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:16 smithi139 ceph-mon[91894]: pgmap v130: 161 pgs: 161 active+clean; 585 KiB data, 153 MiB used, 715 GiB / 715 GiB avail; 59 KiB/s rd, 0 B/s wr, 98 op/s; 0 B/s, 1 objects/s recovering 2024-03-24T11:27:18.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:18 smithi062 ceph-mon[90553]: pgmap v131: 161 pgs: 161 active+clean; 585 KiB data, 153 MiB used, 715 GiB / 715 GiB avail; 49 KiB/s rd, 0 B/s wr, 81 op/s 2024-03-24T11:27:18.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:18 smithi062 ceph-mon[98643]: pgmap v131: 161 pgs: 161 active+clean; 585 KiB data, 153 MiB used, 715 GiB / 715 GiB avail; 49 KiB/s rd, 0 B/s wr, 81 op/s 2024-03-24T11:27:18.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:18 smithi139 ceph-mon[91894]: pgmap v131: 161 pgs: 161 active+clean; 585 KiB data, 153 MiB used, 715 GiB / 715 GiB avail; 49 KiB/s rd, 0 B/s wr, 81 op/s 2024-03-24T11:27:19.346 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:19 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-24T11:27:19.346 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:19 smithi062 ceph-mon[98643]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-24T11:27:19.346 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:19 smithi062 ceph-mon[98643]: Upgrade: osd.2 is safe to restart 2024-03-24T11:27:19.350 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-24T11:27:19.350 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:19 smithi062 ceph-mon[90553]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-24T11:27:19.350 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:19 smithi062 ceph-mon[90553]: Upgrade: osd.2 is safe to restart 2024-03-24T11:27:19.468 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:19 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-24T11:27:19.468 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:19 smithi139 ceph-mon[91894]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-24T11:27:19.468 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:19 smithi139 ceph-mon[91894]: Upgrade: osd.2 is safe to restart 2024-03-24T11:27:20.719 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:20 smithi062 ceph-mon[98643]: pgmap v132: 161 pgs: 161 active+clean; 585 KiB data, 153 MiB used, 715 GiB / 715 GiB avail; 11 KiB/s rd, 0 B/s wr, 18 op/s 2024-03-24T11:27:20.719 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:20.719 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:20.719 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:20.720 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:20.720 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-24T11:27:20.720 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:20.720 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:20 smithi062 ceph-mon[90553]: pgmap v132: 161 pgs: 161 active+clean; 585 KiB data, 153 MiB used, 715 GiB / 715 GiB avail; 11 KiB/s rd, 0 B/s wr, 18 op/s 2024-03-24T11:27:20.720 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:20.720 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:20.720 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:20.720 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:20.720 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-24T11:27:20.720 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:20.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:20 smithi139 ceph-mon[91894]: pgmap v132: 161 pgs: 161 active+clean; 585 KiB data, 153 MiB used, 715 GiB / 715 GiB avail; 11 KiB/s rd, 0 B/s wr, 18 op/s 2024-03-24T11:27:20.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:20.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:20.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:20.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:20.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-24T11:27:20.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:20.977 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:27:20 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:27:20] "GET /metrics HTTP/1.1" 200 38495 "" "Prometheus/2.33.4" 2024-03-24T11:27:21.667 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:21 smithi062 ceph-mon[90553]: Upgrade: Updating osd.2 2024-03-24T11:27:21.667 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:21 smithi062 ceph-mon[90553]: Deploying daemon osd.2 on smithi062 2024-03-24T11:27:21.667 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:21 smithi062 ceph-mon[98643]: Upgrade: Updating osd.2 2024-03-24T11:27:21.668 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:21 smithi062 ceph-mon[98643]: Deploying daemon osd.2 on smithi062 2024-03-24T11:27:21.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:21 smithi139 ceph-mon[91894]: Upgrade: Updating osd.2 2024-03-24T11:27:21.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:21 smithi139 ceph-mon[91894]: Deploying daemon osd.2 on smithi062 2024-03-24T11:27:22.646 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:22 smithi062 ceph-mon[90553]: pgmap v133: 161 pgs: 161 active+clean; 585 KiB data, 153 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:27:22.646 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:22 smithi062 ceph-mon[98643]: pgmap v133: 161 pgs: 161 active+clean; 585 KiB data, 153 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:27:22.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:22 smithi139 ceph-mon[91894]: pgmap v133: 161 pgs: 161 active+clean; 585 KiB data, 153 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:27:23.682 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:23 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:27:23.683 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:23 smithi062 ceph-mon[98643]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:27:23.683 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:23 smithi062 systemd[1]: Stopping Ceph osd.2 for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:27:23.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:23 smithi139 ceph-mon[91894]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:27:23.941 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:23 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2[38778]: 2024-03-24T11:27:23.810+0000 7f7e57cf9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T11:27:23.941 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:23 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2[38778]: 2024-03-24T11:27:23.810+0000 7f7e57cf9700 -1 osd.2 128 *** Got signal Terminated *** 2024-03-24T11:27:23.941 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:23 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2[38778]: 2024-03-24T11:27:23.810+0000 7f7e57cf9700 -1 osd.2 128 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T11:27:24.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:24 smithi062 ceph-mon[90553]: pgmap v134: 161 pgs: 161 active+clean; 585 KiB data, 153 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:27:24.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:24 smithi062 ceph-mon[90553]: osd.2 marked itself down and dead 2024-03-24T11:27:24.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:24.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:24.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:24 smithi062 ceph-mon[90553]: pgmap v135: 161 pgs: 161 active+clean; 585 KiB data, 153 MiB used, 715 GiB / 715 GiB avail; 945 B/s rd, 0 op/s 2024-03-24T11:27:24.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:24.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:24 smithi062 ceph-mon[98643]: pgmap v134: 161 pgs: 161 active+clean; 585 KiB data, 153 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:27:24.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:24 smithi062 ceph-mon[98643]: osd.2 marked itself down and dead 2024-03-24T11:27:24.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:24.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:24.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:24 smithi062 ceph-mon[98643]: pgmap v135: 161 pgs: 161 active+clean; 585 KiB data, 153 MiB used, 715 GiB / 715 GiB avail; 945 B/s rd, 0 op/s 2024-03-24T11:27:24.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:24.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:24 smithi139 ceph-mon[91894]: pgmap v134: 161 pgs: 161 active+clean; 585 KiB data, 153 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:27:24.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:24 smithi139 ceph-mon[91894]: osd.2 marked itself down and dead 2024-03-24T11:27:24.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:24.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:24.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:24 smithi139 ceph-mon[91894]: pgmap v135: 161 pgs: 161 active+clean; 585 KiB data, 153 MiB used, 715 GiB / 715 GiB avail; 945 B/s rd, 0 op/s 2024-03-24T11:27:24.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:25.199 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:24 smithi062 podman[129991]: 2024-03-24 11:27:24.922409764 +0000 UTC m=+1.276634373 container died 714f550103a71b99f99afd4b6c78bfa29828ccaa40f2d545d4ae417f22fed80e (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2, RELEASE=HEAD, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, architecture=x86_64, GIT_REPO=https://github.com/ceph/ceph-container.git, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, io.k8s.display-name=CentOS Stream 8, release=754, maintainer=Guillaume Abrioux , vcs-type=git, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, name=centos-stream, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, build-date=2022-05-03T08:36:31.336870, com.redhat.component=centos-stream-container, vendor=Red Hat, Inc., distribution-scope=public, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., version=8, io.buildah.version=1.19.8, CEPH_POINT_RELEASE=-17.2.0, io.openshift.tags=base centos centos-stream, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, ceph=True, io.openshift.expose-services=) 2024-03-24T11:27:25.468 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:25 smithi062 podman[129991]: 2024-03-24 11:27:25.381058601 +0000 UTC m=+1.735283212 container cleanup 714f550103a71b99f99afd4b6c78bfa29828ccaa40f2d545d4ae417f22fed80e (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2, vcs-type=git, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.openshift.tags=base centos centos-stream, GIT_CLEAN=True, io.buildah.version=1.19.8, io.openshift.expose-services=, com.redhat.component=centos-stream-container, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, build-date=2022-05-03T08:36:31.336870, maintainer=Guillaume Abrioux , io.k8s.display-name=CentOS Stream 8, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, GIT_REPO=https://github.com/ceph/ceph-container.git, architecture=x86_64, RELEASE=HEAD, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, version=8, name=centos-stream, vendor=Red Hat, Inc., GIT_BRANCH=HEAD, distribution-scope=public, release=754, ceph=True, CEPH_POINT_RELEASE=-17.2.0) 2024-03-24T11:27:25.468 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:25 smithi062 bash[129991]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2 2024-03-24T11:27:25.727 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:25 smithi062 podman[130084]: 2024-03-24 11:27:25.666031295 +0000 UTC m=+0.744095336 container remove 714f550103a71b99f99afd4b6c78bfa29828ccaa40f2d545d4ae417f22fed80e (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, ceph=True, maintainer=Guillaume Abrioux , release=754, GIT_REPO=https://github.com/ceph/ceph-container.git, name=centos-stream, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, version=8, GIT_CLEAN=True, vendor=Red Hat, Inc., summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, io.buildah.version=1.19.8, io.openshift.expose-services=, CEPH_POINT_RELEASE=-17.2.0, GIT_BRANCH=HEAD, vcs-type=git, build-date=2022-05-03T08:36:31.336870, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.k8s.display-name=CentOS Stream 8, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, RELEASE=HEAD, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, distribution-scope=public, architecture=x86_64, io.openshift.tags=base centos centos-stream, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, com.redhat.component=centos-stream-container) 2024-03-24T11:27:25.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:25 smithi062 ceph-mon[90553]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T11:27:25.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:25 smithi062 ceph-mon[90553]: failed to encode map e129 with expected crc 2024-03-24T11:27:25.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:25 smithi062 ceph-mon[90553]: failed to encode map e129 with expected crc 2024-03-24T11:27:25.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:25 smithi062 ceph-mon[90553]: osdmap e129: 8 total, 7 up, 8 in 2024-03-24T11:27:25.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:25 smithi062 ceph-mon[90553]: failed to encode map e129 with expected crc 2024-03-24T11:27:25.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:25 smithi062 ceph-mon[90553]: failed to encode map e129 with expected crc 2024-03-24T11:27:25.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:25 smithi062 ceph-mon[90553]: failed to encode map e129 with expected crc 2024-03-24T11:27:25.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:25 smithi062 ceph-mon[90553]: failed to encode map e129 with expected crc 2024-03-24T11:27:25.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:25 smithi062 ceph-mon[90553]: failed to encode map e129 with expected crc 2024-03-24T11:27:25.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:25 smithi062 ceph-mon[90553]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-03-24T11:27:25.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:25 smithi062 ceph-mon[98643]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T11:27:25.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:25 smithi062 ceph-mon[98643]: failed to encode map e129 with expected crc 2024-03-24T11:27:25.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:25 smithi062 ceph-mon[98643]: failed to encode map e129 with expected crc 2024-03-24T11:27:25.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:25 smithi062 ceph-mon[98643]: osdmap e129: 8 total, 7 up, 8 in 2024-03-24T11:27:25.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:25 smithi062 ceph-mon[98643]: failed to encode map e129 with expected crc 2024-03-24T11:27:25.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:25 smithi062 ceph-mon[98643]: failed to encode map e129 with expected crc 2024-03-24T11:27:25.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:25 smithi062 ceph-mon[98643]: failed to encode map e129 with expected crc 2024-03-24T11:27:25.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:25 smithi062 ceph-mon[98643]: failed to encode map e129 with expected crc 2024-03-24T11:27:25.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:25 smithi062 ceph-mon[98643]: failed to encode map e129 with expected crc 2024-03-24T11:27:25.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:25 smithi062 ceph-mon[98643]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-03-24T11:27:25.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:25 smithi139 ceph-mon[91894]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T11:27:25.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:25 smithi139 ceph-mon[91894]: failed to encode map e129 with expected crc 2024-03-24T11:27:25.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:25 smithi139 ceph-mon[91894]: failed to encode map e129 with expected crc 2024-03-24T11:27:25.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:25 smithi139 ceph-mon[91894]: osdmap e129: 8 total, 7 up, 8 in 2024-03-24T11:27:25.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:25 smithi139 ceph-mon[91894]: failed to encode map e129 with expected crc 2024-03-24T11:27:25.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:25 smithi139 ceph-mon[91894]: failed to encode map e129 with expected crc 2024-03-24T11:27:25.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:25 smithi139 ceph-mon[91894]: failed to encode map e129 with expected crc 2024-03-24T11:27:25.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:25 smithi139 ceph-mon[91894]: failed to encode map e129 with expected crc 2024-03-24T11:27:25.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:25 smithi139 ceph-mon[91894]: failed to encode map e129 with expected crc 2024-03-24T11:27:25.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:25 smithi139 ceph-mon[91894]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-03-24T11:27:26.295 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:26 smithi062 podman[130172]: 2024-03-24 11:27:25.920438763 +0000 UTC m=+0.018182136 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:27:26.295 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:26 smithi062 podman[130172]: 2024-03-24 11:27:26.042056805 +0000 UTC m=+0.139800141 container create e0301d85e3287005968cc2aa5f5f8abd58412223f4efdf6c30755942a1db4b94 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-db0330b, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-24T11:27:26.598 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:26 smithi062 podman[130172]: 2024-03-24 11:27:26.295067616 +0000 UTC m=+0.392810977 container init e0301d85e3287005968cc2aa5f5f8abd58412223f4efdf6c30755942a1db4b94 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-db0330b, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T11:27:26.598 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:26 smithi062 podman[130172]: 2024-03-24 11:27:26.298417597 +0000 UTC m=+0.396160942 container start e0301d85e3287005968cc2aa5f5f8abd58412223f4efdf6c30755942a1db4b94 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-db0330b, GIT_BRANCH=HEAD) 2024-03-24T11:27:26.598 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:26 smithi062 podman[130172]: 2024-03-24 11:27:26.386890421 +0000 UTC m=+0.484633786 container attach e0301d85e3287005968cc2aa5f5f8abd58412223f4efdf6c30755942a1db4b94 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2-deactivate, RELEASE=squid-db0330b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-24T11:27:26.598 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:26 smithi062 ceph-mon[90553]: failed to encode map e130 with expected crc 2024-03-24T11:27:26.599 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:26 smithi062 ceph-mon[90553]: failed to encode map e130 with expected crc 2024-03-24T11:27:26.599 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:26 smithi062 ceph-mon[90553]: osdmap e130: 8 total, 7 up, 8 in 2024-03-24T11:27:26.599 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:26 smithi062 ceph-mon[90553]: failed to encode map e130 with expected crc 2024-03-24T11:27:26.599 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:26 smithi062 ceph-mon[90553]: failed to encode map e130 with expected crc 2024-03-24T11:27:26.599 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:26 smithi062 ceph-mon[90553]: failed to encode map e130 with expected crc 2024-03-24T11:27:26.599 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:26 smithi062 ceph-mon[90553]: failed to encode map e130 with expected crc 2024-03-24T11:27:26.599 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:26 smithi062 ceph-mon[90553]: failed to encode map e130 with expected crc 2024-03-24T11:27:26.599 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:26 smithi062 ceph-mon[90553]: failed to encode map e130 with expected crc 2024-03-24T11:27:26.599 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:26 smithi062 ceph-mon[90553]: pgmap v138: 161 pgs: 13 stale+active+clean, 148 active+clean; 585 KiB data, 153 MiB used, 715 GiB / 715 GiB avail; 749 B/s rd, 0 op/s 2024-03-24T11:27:26.599 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:26.599 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:27:26.599 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:26 smithi062 ceph-mon[98643]: failed to encode map e130 with expected crc 2024-03-24T11:27:26.600 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:26 smithi062 ceph-mon[98643]: failed to encode map e130 with expected crc 2024-03-24T11:27:26.600 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:26 smithi062 ceph-mon[98643]: osdmap e130: 8 total, 7 up, 8 in 2024-03-24T11:27:26.600 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:26 smithi062 ceph-mon[98643]: failed to encode map e130 with expected crc 2024-03-24T11:27:26.600 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:26 smithi062 ceph-mon[98643]: failed to encode map e130 with expected crc 2024-03-24T11:27:26.600 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:26 smithi062 ceph-mon[98643]: failed to encode map e130 with expected crc 2024-03-24T11:27:26.600 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:26 smithi062 ceph-mon[98643]: failed to encode map e130 with expected crc 2024-03-24T11:27:26.600 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:26 smithi062 ceph-mon[98643]: failed to encode map e130 with expected crc 2024-03-24T11:27:26.600 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:26 smithi062 ceph-mon[98643]: failed to encode map e130 with expected crc 2024-03-24T11:27:26.600 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:26 smithi062 ceph-mon[98643]: pgmap v138: 161 pgs: 13 stale+active+clean, 148 active+clean; 585 KiB data, 153 MiB used, 715 GiB / 715 GiB avail; 749 B/s rd, 0 op/s 2024-03-24T11:27:26.600 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:26.600 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:27:26.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:26 smithi139 ceph-mon[91894]: failed to encode map e130 with expected crc 2024-03-24T11:27:26.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:26 smithi139 ceph-mon[91894]: failed to encode map e130 with expected crc 2024-03-24T11:27:26.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:26 smithi139 ceph-mon[91894]: osdmap e130: 8 total, 7 up, 8 in 2024-03-24T11:27:26.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:26 smithi139 ceph-mon[91894]: failed to encode map e130 with expected crc 2024-03-24T11:27:26.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:26 smithi139 ceph-mon[91894]: failed to encode map e130 with expected crc 2024-03-24T11:27:26.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:26 smithi139 ceph-mon[91894]: failed to encode map e130 with expected crc 2024-03-24T11:27:26.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:26 smithi139 ceph-mon[91894]: failed to encode map e130 with expected crc 2024-03-24T11:27:26.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:26 smithi139 ceph-mon[91894]: failed to encode map e130 with expected crc 2024-03-24T11:27:26.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:26 smithi139 ceph-mon[91894]: failed to encode map e130 with expected crc 2024-03-24T11:27:26.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:26 smithi139 ceph-mon[91894]: pgmap v138: 161 pgs: 13 stale+active+clean, 148 active+clean; 585 KiB data, 153 MiB used, 715 GiB / 715 GiB avail; 749 B/s rd, 0 op/s 2024-03-24T11:27:26.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:26.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:27:26.932 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:26 smithi062 podman[130172]: 2024-03-24 11:27:26.598222526 +0000 UTC m=+0.695965864 container died e0301d85e3287005968cc2aa5f5f8abd58412223f4efdf6c30755942a1db4b94 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.schema-version=1.0, RELEASE=squid-db0330b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-03-24T11:27:27.582 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:27 smithi062 podman[130199]: 2024-03-24 11:27:27.324690876 +0000 UTC m=+0.721290448 container remove e0301d85e3287005968cc2aa5f5f8abd58412223f4efdf6c30755942a1db4b94 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T11:27:27.582 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:27 smithi062 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.2.service: Deactivated successfully. 2024-03-24T11:27:27.582 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:27 smithi062 systemd[1]: Stopped Ceph osd.2 for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:27:27.582 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:27 smithi062 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.2.service: Consumed 7.773s CPU time. 2024-03-24T11:27:27.870 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:27 smithi062 systemd[1]: Starting Ceph osd.2 for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:27:28.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:28 smithi062 ceph-mon[90553]: pgmap v139: 161 pgs: 11 active+undersized+degraded, 4 stale+active+clean, 21 active+undersized, 125 active+clean; 585 KiB data, 154 MiB used, 715 GiB / 715 GiB avail; 32/630 objects degraded (5.079%) 2024-03-24T11:27:28.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:28 smithi062 ceph-mon[90553]: Health check failed: Degraded data redundancy: 32/630 objects degraded (5.079%), 11 pgs degraded (PG_DEGRADED) 2024-03-24T11:27:28.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:28 smithi062 ceph-mon[98643]: pgmap v139: 161 pgs: 11 active+undersized+degraded, 4 stale+active+clean, 21 active+undersized, 125 active+clean; 585 KiB data, 154 MiB used, 715 GiB / 715 GiB avail; 32/630 objects degraded (5.079%) 2024-03-24T11:27:28.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:28 smithi062 ceph-mon[98643]: Health check failed: Degraded data redundancy: 32/630 objects degraded (5.079%), 11 pgs degraded (PG_DEGRADED) 2024-03-24T11:27:28.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:28 smithi139 ceph-mon[91894]: pgmap v139: 161 pgs: 11 active+undersized+degraded, 4 stale+active+clean, 21 active+undersized, 125 active+clean; 585 KiB data, 154 MiB used, 715 GiB / 715 GiB avail; 32/630 objects degraded (5.079%) 2024-03-24T11:27:28.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:28 smithi139 ceph-mon[91894]: Health check failed: Degraded data redundancy: 32/630 objects degraded (5.079%), 11 pgs degraded (PG_DEGRADED) 2024-03-24T11:27:28.522 INFO:teuthology.orchestra.run.smithi062.stdout:true 2024-03-24T11:27:28.855 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:28 smithi062 podman[130364]: 2024-03-24 11:27:28.599556127 +0000 UTC m=+0.018124735 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:27:28.855 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:28 smithi062 podman[130364]: 2024-03-24 11:27:28.749695683 +0000 UTC m=+0.168264288 container create 70643dc0c8f97d17394223f6bb3f504fa2d4a269ee2e1c4949c7f43d26741e27 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-db0330b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-24T11:27:29.209 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:29 smithi062 ceph-mon[90553]: from='client.64306 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:27:29.210 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:29 smithi062 ceph-mon[90553]: from='client.44346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:27:29.210 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:29 smithi062 ceph-mon[98643]: from='client.64306 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:27:29.210 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:29 smithi062 ceph-mon[98643]: from='client.44346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:27:29.210 INFO:teuthology.orchestra.run.smithi062.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T11:27:29.210 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi062 smithi062 running 5s ago 16m - - 2024-03-24T11:27:29.211 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi139 smithi139 running 9s ago 16m - - 2024-03-24T11:27:29.211 INFO:teuthology.orchestra.run.smithi062.stdout:alertmanager.a smithi062 *:9093,9094 running (6m) 5s ago 11m 24.6M - 0.23.0 ba2b418f427c 818298d58140 2024-03-24T11:27:29.211 INFO:teuthology.orchestra.run.smithi062.stdout:grafana.a smithi139 *:3000 running (6m) 9s ago 11m 53.8M - 8.3.5 dad864ee21e9 b48167e7171e 2024-03-24T11:27:29.211 INFO:teuthology.orchestra.run.smithi062.stdout:iscsi.foo.smithi062.dggsvk smithi062 running (5m) 5s ago 10m 58.6M - 3.5 e1d6a67b021e 0a98806ec839 2024-03-24T11:27:29.211 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.x smithi139 *:8443,9283,8765 running (4m) 9s ago 16m 451M - 19.0.0-1667-gdb0330b1 4f325f8440e9 d94987b11387 2024-03-24T11:27:29.211 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.y smithi062 *:8443,9283,8765 running (5m) 5s ago 17m 524M - 19.0.0-1667-gdb0330b1 4f325f8440e9 ca1a49028d65 2024-03-24T11:27:29.211 INFO:teuthology.orchestra.run.smithi062.stdout:mon.a smithi062 running (3m) 5s ago 17m 59.0M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 f7afe6e28c88 2024-03-24T11:27:29.211 INFO:teuthology.orchestra.run.smithi062.stdout:mon.b smithi139 running (2m) 9s ago 16m 41.0M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 ca357beea4dd 2024-03-24T11:27:29.211 INFO:teuthology.orchestra.run.smithi062.stdout:mon.c smithi062 running (2m) 5s ago 16m 49.9M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 459e7449ec39 2024-03-24T11:27:29.211 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.a smithi062 *:9100 running (6m) 5s ago 12m 17.2M - 1.3.1 1dbe0e931976 79d5a02a999f 2024-03-24T11:27:29.211 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.b smithi139 *:9100 running (6m) 9s ago 12m 15.9M - 1.3.1 1dbe0e931976 8f9f4608e465 2024-03-24T11:27:29.211 INFO:teuthology.orchestra.run.smithi062.stdout:osd.0 smithi062 running (84s) 5s ago 15m 53.3M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 dfe4920b49c4 2024-03-24T11:27:29.211 INFO:teuthology.orchestra.run.smithi062.stdout:osd.1 smithi062 running (35s) 5s ago 15m 44.1M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 c443b3fe97c1 2024-03-24T11:27:29.211 INFO:teuthology.orchestra.run.smithi062.stdout:osd.2 smithi062 unknown 5s ago 15m 55.3M 4096M 17.2.0 e1d6a67b021e 714f550103a7 2024-03-24T11:27:29.212 INFO:teuthology.orchestra.run.smithi062.stdout:osd.3 smithi062 running (14m) 5s ago 14m 61.4M 4096M 17.2.0 e1d6a67b021e 1d2868bd5195 2024-03-24T11:27:29.212 INFO:teuthology.orchestra.run.smithi062.stdout:osd.4 smithi139 running (14m) 9s ago 14m 65.2M 4096M 17.2.0 e1d6a67b021e 7568bfa9e74d 2024-03-24T11:27:29.212 INFO:teuthology.orchestra.run.smithi062.stdout:osd.5 smithi139 running (13m) 9s ago 13m 58.7M 4096M 17.2.0 e1d6a67b021e dd0abd599a14 2024-03-24T11:27:29.212 INFO:teuthology.orchestra.run.smithi062.stdout:osd.6 smithi139 running (13m) 9s ago 13m 57.9M 4096M 17.2.0 e1d6a67b021e a4b1289aca69 2024-03-24T11:27:29.212 INFO:teuthology.orchestra.run.smithi062.stdout:osd.7 smithi139 running (13m) 9s ago 13m 59.9M 4096M 17.2.0 e1d6a67b021e bc7fadbc1384 2024-03-24T11:27:29.212 INFO:teuthology.orchestra.run.smithi062.stdout:prometheus.a smithi139 *:9095 running (5m) 9s ago 12m 71.1M - 2.33.4 514e6a882f6e 2ed47f810d33 2024-03-24T11:27:29.212 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi062.sacpds smithi062 *:8000 running (10m) 5s ago 10m 73.9M - 17.2.0 e1d6a67b021e 89f4ac308e98 2024-03-24T11:27:29.212 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi139.yljesh smithi139 *:8000 running (10m) 9s ago 10m 72.9M - 17.2.0 e1d6a67b021e e189bdc12cae 2024-03-24T11:27:29.212 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi062.xvpxnk smithi062 *:80 running (10m) 5s ago 10m 73.9M - 17.2.0 e1d6a67b021e d1aa74adab44 2024-03-24T11:27:29.212 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi139.pbtrmp smithi139 *:80 running (10m) 9s ago 10m 76.6M - 17.2.0 e1d6a67b021e 7f8e336755d7 2024-03-24T11:27:29.212 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:28 smithi062 podman[130364]: 2024-03-24 11:27:28.993885935 +0000 UTC m=+0.412454541 container init 70643dc0c8f97d17394223f6bb3f504fa2d4a269ee2e1c4949c7f43d26741e27 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2-activate, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-24T11:27:29.212 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:28 smithi062 podman[130364]: 2024-03-24 11:27:28.997153091 +0000 UTC m=+0.415721709 container start 70643dc0c8f97d17394223f6bb3f504fa2d4a269ee2e1c4949c7f43d26741e27 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2-activate, RELEASE=squid-db0330b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T11:27:29.213 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:29 smithi062 podman[130364]: 2024-03-24 11:27:29.094292847 +0000 UTC m=+0.512861466 container attach 70643dc0c8f97d17394223f6bb3f504fa2d4a269ee2e1c4949c7f43d26741e27 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2-activate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-03-24T11:27:29.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:29 smithi139 ceph-mon[91894]: from='client.64306 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:27:29.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:29 smithi139 ceph-mon[91894]: from='client.44346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:27:29.477 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:29 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2-activate[130399]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-24T11:27:29.477 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:29 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2-activate[130399]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-24T11:27:29.477 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:29 smithi062 bash[130364]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-24T11:27:29.478 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:29 smithi062 bash[130364]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-24T11:27:29.478 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:29 smithi062 bash[130364]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-24T11:27:29.478 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:29 smithi062 bash[130364]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-24T11:27:29.478 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:29 smithi062 bash[130364]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-03-24T11:27:29.478 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:29 smithi062 bash[130364]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-24T11:27:29.478 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:29 smithi062 bash[130364]: --> ceph-volume raw activate successful for osd ID: 2 2024-03-24T11:27:29.478 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:29 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2-activate[130399]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-24T11:27:29.478 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:29 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2-activate[130399]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-24T11:27:29.478 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:29 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2-activate[130399]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-03-24T11:27:29.478 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:29 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2-activate[130399]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-24T11:27:29.478 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:29 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2-activate[130399]: --> ceph-volume raw activate successful for osd ID: 2 2024-03-24T11:27:29.639 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:27:29.639 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": { 2024-03-24T11:27:29.639 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 3 2024-03-24T11:27:29.639 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:27:29.639 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": { 2024-03-24T11:27:29.639 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-24T11:27:29.639 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:27:29.639 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": { 2024-03-24T11:27:29.640 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 5, 2024-03-24T11:27:29.640 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-24T11:27:29.640 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:27:29.640 INFO:teuthology.orchestra.run.smithi062.stdout: "rgw": { 2024-03-24T11:27:29.640 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4 2024-03-24T11:27:29.640 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:27:29.640 INFO:teuthology.orchestra.run.smithi062.stdout: "overall": { 2024-03-24T11:27:29.640 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 9, 2024-03-24T11:27:29.640 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 7 2024-03-24T11:27:29.640 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-24T11:27:29.640 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:27:29.773 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:29 smithi062 podman[130364]: 2024-03-24 11:27:29.484055518 +0000 UTC m=+0.902624124 container died 70643dc0c8f97d17394223f6bb3f504fa2d4a269ee2e1c4949c7f43d26741e27 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2-activate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=squid-db0330b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True) 2024-03-24T11:27:29.982 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:27:29.982 INFO:teuthology.orchestra.run.smithi062.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69", 2024-03-24T11:27:29.982 INFO:teuthology.orchestra.run.smithi062.stdout: "in_progress": true, 2024-03-24T11:27:29.982 INFO:teuthology.orchestra.run.smithi062.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T11:27:29.982 INFO:teuthology.orchestra.run.smithi062.stdout: "services_complete": [ 2024-03-24T11:27:29.982 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr", 2024-03-24T11:27:29.983 INFO:teuthology.orchestra.run.smithi062.stdout: "mon" 2024-03-24T11:27:29.983 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-03-24T11:27:29.983 INFO:teuthology.orchestra.run.smithi062.stdout: "progress": "7/23 daemons upgraded", 2024-03-24T11:27:29.983 INFO:teuthology.orchestra.run.smithi062.stdout: "message": "Currently upgrading osd daemons", 2024-03-24T11:27:29.983 INFO:teuthology.orchestra.run.smithi062.stdout: "is_paused": false 2024-03-24T11:27:29.983 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:27:30.307 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:30 smithi062 ceph-mon[98643]: from='client.44352 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:27:30.307 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:30 smithi062 ceph-mon[98643]: from='client.? 172.21.15.62:0/2391170337' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:30.307 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:30 smithi062 ceph-mon[98643]: pgmap v140: 161 pgs: 14 active+undersized+degraded, 32 active+undersized, 115 active+clean; 585 KiB data, 154 MiB used, 715 GiB / 715 GiB avail; 56/630 objects degraded (8.889%) 2024-03-24T11:27:30.307 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:30 smithi062 ceph-mon[98643]: from='client.44364 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:27:30.308 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:30 smithi062 ceph-mon[90553]: from='client.44352 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:27:30.308 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:30 smithi062 ceph-mon[90553]: from='client.? 172.21.15.62:0/2391170337' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:30.308 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:30 smithi062 ceph-mon[90553]: pgmap v140: 161 pgs: 14 active+undersized+degraded, 32 active+undersized, 115 active+clean; 585 KiB data, 154 MiB used, 715 GiB / 715 GiB avail; 56/630 objects degraded (8.889%) 2024-03-24T11:27:30.308 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:30 smithi062 ceph-mon[90553]: from='client.44364 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:27:30.308 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:30 smithi062 podman[130566]: 2024-03-24 11:27:30.090372967 +0000 UTC m=+0.599899045 container remove 70643dc0c8f97d17394223f6bb3f504fa2d4a269ee2e1c4949c7f43d26741e27 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2-activate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240307, GIT_CLEAN=True, RELEASE=squid-db0330b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T11:27:30.308 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:30 smithi062 podman[130676]: 2024-03-24 11:27:30.291473387 +0000 UTC m=+0.099706153 container create da9e6fa160245eef870511413321664d0b1aaa4d125f560a52aeea31ac984100 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-24T11:27:30.308 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:30 smithi062 podman[130676]: 2024-03-24 11:27:30.210789584 +0000 UTC m=+0.019022357 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:27:30.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:30 smithi139 ceph-mon[91894]: from='client.44352 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:27:30.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:30 smithi139 ceph-mon[91894]: from='client.? 172.21.15.62:0/2391170337' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:30.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:30 smithi139 ceph-mon[91894]: pgmap v140: 161 pgs: 14 active+undersized+degraded, 32 active+undersized, 115 active+clean; 585 KiB data, 154 MiB used, 715 GiB / 715 GiB avail; 56/630 objects degraded (8.889%) 2024-03-24T11:27:30.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:30 smithi139 ceph-mon[91894]: from='client.44364 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:27:30.405 INFO:teuthology.orchestra.run.smithi062.stdout:HEALTH_WARN 1 failed cephadm daemon(s); 1 osds down; Degraded data redundancy: 56/630 objects degraded (8.889%), 14 pgs degraded 2024-03-24T11:27:30.405 INFO:teuthology.orchestra.run.smithi062.stdout:[WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-03-24T11:27:30.405 INFO:teuthology.orchestra.run.smithi062.stdout: daemon osd.2 on smithi062 is in unknown state 2024-03-24T11:27:30.406 INFO:teuthology.orchestra.run.smithi062.stdout:[WRN] OSD_DOWN: 1 osds down 2024-03-24T11:27:30.406 INFO:teuthology.orchestra.run.smithi062.stdout: osd.2 (root=default,host=smithi062) is down 2024-03-24T11:27:30.406 INFO:teuthology.orchestra.run.smithi062.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 56/630 objects degraded (8.889%), 14 pgs degraded 2024-03-24T11:27:30.406 INFO:teuthology.orchestra.run.smithi062.stdout: pg 2.3 is active+undersized+degraded, acting [5,7] 2024-03-24T11:27:30.406 INFO:teuthology.orchestra.run.smithi062.stdout: pg 2.e is active+undersized+degraded, acting [3,7] 2024-03-24T11:27:30.406 INFO:teuthology.orchestra.run.smithi062.stdout: pg 2.16 is active+undersized+degraded, acting [5,6] 2024-03-24T11:27:30.406 INFO:teuthology.orchestra.run.smithi062.stdout: pg 3.0 is active+undersized+degraded, acting [1,6] 2024-03-24T11:27:30.406 INFO:teuthology.orchestra.run.smithi062.stdout: pg 3.4 is active+undersized+degraded, acting [1,5] 2024-03-24T11:27:30.406 INFO:teuthology.orchestra.run.smithi062.stdout: pg 3.5 is active+undersized+degraded, acting [5,3] 2024-03-24T11:27:30.406 INFO:teuthology.orchestra.run.smithi062.stdout: pg 3.9 is active+undersized+degraded, acting [4,7] 2024-03-24T11:27:30.406 INFO:teuthology.orchestra.run.smithi062.stdout: pg 3.13 is active+undersized+degraded, acting [7,4] 2024-03-24T11:27:30.406 INFO:teuthology.orchestra.run.smithi062.stdout: pg 3.1a is active+undersized+degraded, acting [4,1] 2024-03-24T11:27:30.406 INFO:teuthology.orchestra.run.smithi062.stdout: pg 3.1e is active+undersized+degraded, acting [3,6] 2024-03-24T11:27:30.407 INFO:teuthology.orchestra.run.smithi062.stdout: pg 3.1f is active+undersized+degraded, acting [0,5] 2024-03-24T11:27:30.407 INFO:teuthology.orchestra.run.smithi062.stdout: pg 4.d is active+undersized+degraded, acting [4,1] 2024-03-24T11:27:30.407 INFO:teuthology.orchestra.run.smithi062.stdout: pg 5.6 is active+undersized+degraded, acting [5,7] 2024-03-24T11:27:30.407 INFO:teuthology.orchestra.run.smithi062.stdout: pg 5.1c is active+undersized+degraded, acting [4,3] 2024-03-24T11:27:30.577 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:30 smithi062 podman[130676]: 2024-03-24 11:27:30.51912112 +0000 UTC m=+0.327353896 container init da9e6fa160245eef870511413321664d0b1aaa4d125f560a52aeea31ac984100 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-db0330b, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS) 2024-03-24T11:27:30.577 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:30 smithi062 podman[130676]: 2024-03-24 11:27:30.522049999 +0000 UTC m=+0.330282768 container start da9e6fa160245eef870511413321664d0b1aaa4d125f560a52aeea31ac984100 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-db0330b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T11:27:30.578 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:30 smithi062 ceph-osd[130698]: -- 172.21.15.62:0/3727164780 <== mon.0 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5631ba9711e0 con 0x5631ba943400 2024-03-24T11:27:30.921 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:27:30 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:27:30] "GET /metrics HTTP/1.1" 200 38501 "" "Prometheus/2.33.4" 2024-03-24T11:27:30.922 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:30 smithi062 bash[130676]: da9e6fa160245eef870511413321664d0b1aaa4d125f560a52aeea31ac984100 2024-03-24T11:27:30.922 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:30 smithi062 systemd[1]: Started Ceph osd.2 for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:27:31.051 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:31 smithi139 ceph-mon[91894]: from='client.? 172.21.15.62:0/174170778' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:27:31.052 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:31.052 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:31.052 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:31.052 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:31.052 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:31 smithi139 ceph-mon[91894]: pgmap v141: 161 pgs: 14 active+undersized+degraded, 32 active+undersized, 115 active+clean; 585 KiB data, 154 MiB used, 715 GiB / 715 GiB avail; 56/630 objects degraded (8.889%) 2024-03-24T11:27:31.052 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:31.052 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:31 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:31.052 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:31.052 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:31.052 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:31.052 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:31.052 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:31.052 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:31 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:31.052 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:31.052 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:31.053 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:31.053 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:31.053 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:31.053 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:31 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:31.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:31 smithi062 ceph-mon[90553]: from='client.? 172.21.15.62:0/174170778' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:27:31.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:31.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:31.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:31.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:31.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:31 smithi062 ceph-mon[90553]: pgmap v141: 161 pgs: 14 active+undersized+degraded, 32 active+undersized, 115 active+clean; 585 KiB data, 154 MiB used, 715 GiB / 715 GiB avail; 56/630 objects degraded (8.889%) 2024-03-24T11:27:31.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:31.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:31 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:31.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:31.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:31.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:31.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:31.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:31.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:31 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:31.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:31.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:31.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:31.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:31.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:31.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:31 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:31.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:31 smithi062 ceph-mon[98643]: from='client.? 172.21.15.62:0/174170778' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:27:31.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:31.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:31.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:31.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:31.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:31 smithi062 ceph-mon[98643]: pgmap v141: 161 pgs: 14 active+undersized+degraded, 32 active+undersized, 115 active+clean; 585 KiB data, 154 MiB used, 715 GiB / 715 GiB avail; 56/630 objects degraded (8.889%) 2024-03-24T11:27:31.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:31.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:31 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:31.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:31.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:31.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:31.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:31.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:31.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:31 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:31.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:31.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:31.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:31.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:31.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:31.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:31 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:31.230 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:31 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2[130691]: 2024-03-24T11:27:31.057+0000 7f0bc33337c0 -1 Falling back to public interface 2024-03-24T11:27:32.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:32.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:32.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:32.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:32.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:32.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:32 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:32.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:32.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:32.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:32.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:32.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:32.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:32 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:32.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:32.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:32.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:32.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:32.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:32.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:32 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:32.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:32.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:32.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:32.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:32.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:32.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:32 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:32.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:32 smithi139 ceph-mon[91894]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-03-24T11:27:32.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:32.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:32.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:32.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:32.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:32.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:32 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:32.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:32.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:32.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:32.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:32.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:32.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:32 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:32.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:32.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:32.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:32.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:32.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:32.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:32 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:32.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:32.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:32.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:32.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:32.365 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:32.365 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:32 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:32.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:32.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:32.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:32.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:32.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:32.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:32.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:32.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:32.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:32.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:32.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:32.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:32.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:32.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:32.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:32.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:32.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:32.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:32.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:32.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:32.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:32.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:32.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:32.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:32.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[90553]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-03-24T11:27:32.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:32.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:32.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:32.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:32.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:32.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:32.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:32.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:32.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:32.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:32.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:32.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:32.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:32.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:32.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:32.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:32.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:32.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:32.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:32.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:32.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:32.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:32.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:32.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:32.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:32.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:32.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:32.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:32.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[98643]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-03-24T11:27:32.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:32.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:32.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:32.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:32.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:32.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:32.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:32.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:32.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:32.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:32.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:32.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:32.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:32.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:32.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:32.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:32.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:32.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:32.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:32.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:32.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:32.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:32.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:32.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:32.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:32.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:32.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:32.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:32.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:32.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:32.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:32.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:32.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:32.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:32.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:32.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:32.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:32.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:32.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:32.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:32.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:32.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:32.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:32.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:32 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:33.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:33.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:33.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:33.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:33.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:33.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:33.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:33.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:33.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:33.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:33.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:33.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:33.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:33.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:33.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:33.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:33.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:33.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:33.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:27:33.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:33.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:33.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:33.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:33.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:33.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:33.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:33.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:33.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:33.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:33.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:33.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:33.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:33.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:33.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:33.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:33.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:33.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:33.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:33.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:33.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:33.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:33.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:33.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:33.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:33.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:33.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:33.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:27:33.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:33.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:33.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:33.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:33.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:33.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:33.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:33.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:33.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:33.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:33.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:33.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:33.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:33.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:33.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:33.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:33.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:33.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:33.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:33.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:33.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:33.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:33.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:33.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:33.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: pgmap v142: 161 pgs: 14 active+undersized+degraded, 32 active+undersized, 115 active+clean; 585 KiB data, 154 MiB used, 715 GiB / 715 GiB avail; 56/630 objects degraded (8.889%) 2024-03-24T11:27:33.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:33.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:33.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:33.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:33.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:33.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:33.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:33.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:33.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:33.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:33.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:33.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:33.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:33.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:33.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:33.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:33.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:33.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:33.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:33.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:33.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:33.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:33.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:33.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:33.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:33.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:33.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:33.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:33.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: pgmap v142: 161 pgs: 14 active+undersized+degraded, 32 active+undersized, 115 active+clean; 585 KiB data, 154 MiB used, 715 GiB / 715 GiB avail; 56/630 objects degraded (8.889%) 2024-03-24T11:27:33.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:33.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:33.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:33.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:33.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:33.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:33.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:33.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:33.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:33.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:33.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:33.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:33.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:33.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:33.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:33.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:33.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:33.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:33.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:33.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:33.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:33.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:33.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:33.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:33.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:33.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:33.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:33.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:33.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:33.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:33.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:33.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:33.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:33.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:33.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:33.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:33.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:33.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:33.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:33.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:33.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:33.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:33.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:33.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:33.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:33.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:33.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:33.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:33.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:33.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:33.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:33.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:33.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:33.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:27:33.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:33.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:33.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:33.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:33.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:33.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:33.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:33.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:33.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:33.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:33.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:33.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:33.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:33.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:33.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:33.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:33.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:33.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:33.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:33.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:33.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:33.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:33.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:33.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:33.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: pgmap v142: 161 pgs: 14 active+undersized+degraded, 32 active+undersized, 115 active+clean; 585 KiB data, 154 MiB used, 715 GiB / 715 GiB avail; 56/630 objects degraded (8.889%) 2024-03-24T11:27:33.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:33.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:33.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:33.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:33.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:33.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:33.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:33.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:33.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:33.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:33.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:33.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:33.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:33.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:33.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:33.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:33.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:33.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:33.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:33.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:33.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:33.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:33.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:33.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:33 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:34.052 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2[130691]: 2024-03-24T11:27:33.790+0000 7f0bc33337c0 -1 osd.2 0 read_superblock omap replica is missing. 2024-03-24T11:27:34.052 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2[130691]: 2024-03-24T11:27:33.823+0000 7f0bc33337c0 -1 osd.2 128 log_to_monitors true 2024-03-24T11:27:34.054 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:34.054 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:34.054 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:34.054 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:34.054 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:34.054 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:34.055 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:34.055 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:34.055 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:34.055 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:34.055 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:34.055 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:34.055 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:34.055 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:34.055 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:34.055 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:34.055 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:34.055 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:34.055 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:34.055 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:34.055 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:34.056 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:34.056 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:34.056 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:34.056 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:34.056 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:34.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:34.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:34.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:34.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:34.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:34.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:34.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:34.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:34.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:34.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:34.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:34.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:34.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:34.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:34.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:34.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:34.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:34.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:34.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:34.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:34.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:34.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:34.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:34.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:34.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:34.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:34.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:34.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:34.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:34.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:34.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:34.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:34.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:34.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:34.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:34.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:34.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:34.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:34.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:34.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:34.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:34.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:34.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:34.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:34.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:34.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:34.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:34.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:34.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:34.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:34.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:34.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:34.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:34.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:34.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='osd.2 [v2:172.21.15.62:6818/904171279,v1:172.21.15.62:6819/904171279]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-24T11:27:34.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-24T11:27:34.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:34.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:34.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:34.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:34.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:34.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:34.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:34.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:34.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:34.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:34.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:34.364 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:34.364 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:34.364 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:34.364 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:34.364 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:34.364 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:34.364 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:34.364 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:34.364 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:34.364 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:34.364 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:34.364 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:34.364 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:34.364 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:34.364 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:34.365 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:34.365 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:34.365 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:34.365 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:34.365 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:34.365 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:34.365 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:34.365 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:34.365 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:34.365 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:34.365 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:34.365 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:34.365 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:34.365 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='osd.2 [v2:172.21.15.62:6818/904171279,v1:172.21.15.62:6819/904171279]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-24T11:27:34.366 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:34.366 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:34.366 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:34.366 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:34.366 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:34.366 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:34.366 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:34.366 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:34.366 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:34.366 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:34.366 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:34.367 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:34.367 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:34.367 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:34.367 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:34.367 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:34.367 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:34.367 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:34.367 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:34.367 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:34.367 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:34.367 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:34.367 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:34.367 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:34.367 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:34.367 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:34.368 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:34.368 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:34.368 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:34.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-24T11:27:34.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:34.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:34.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:34.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:34.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:34.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:34.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:34.368 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:34.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:34.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:34.369 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:34.369 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:34.369 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:34.369 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:34.369 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:34.369 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:34.370 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:34.370 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:34.370 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:34.370 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:34.370 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:34.370 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:34.370 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:34.370 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:34.370 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:34.370 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:34.370 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:34.370 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:34.370 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:34.370 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:34.370 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:34.371 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:34.371 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:34.371 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:34.371 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:34.371 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:34.371 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='osd.2 [v2:172.21.15.62:6818/904171279,v1:172.21.15.62:6819/904171279]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-24T11:27:34.371 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-24T11:27:34.371 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:34.371 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:34.371 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:34.371 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:34.371 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:34.371 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:34.371 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:34.372 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:34.372 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:34.372 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:34.372 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:35.477 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2[130691]: 2024-03-24T11:27:35.251+0000 7f0bb0783700 -1 osd.2 128 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T11:27:35.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:35.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:35.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:35.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:35.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:35.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:35.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:35.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: Health check update: Degraded data redundancy: 56/630 objects degraded (8.889%), 14 pgs degraded (PG_DEGRADED) 2024-03-24T11:27:35.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:35.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:35.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:35.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:35.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:35.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:35.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:35.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:35.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:35.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:35.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:35.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:35.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:35.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:35.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:35.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:35.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:35.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:35.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:35.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:35.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:35.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:35.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:35.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:35.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:35.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:35.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:35.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:35.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: Health check update: Degraded data redundancy: 56/630 objects degraded (8.889%), 14 pgs degraded (PG_DEGRADED) 2024-03-24T11:27:35.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:35.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:35.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:35.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:35.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:35.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:35.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:35.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:35.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:35.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:35.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:35.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:35.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:35.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:35.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:35.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:35.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:35.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:35.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:35.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:35.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:35.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:35.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:35.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:35.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:35.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:35.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:35.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:35.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:35.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:35.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: pgmap v143: 161 pgs: 14 active+undersized+degraded, 32 active+undersized, 115 active+clean; 585 KiB data, 154 MiB used, 715 GiB / 715 GiB avail; 56/630 objects degraded (8.889%) 2024-03-24T11:27:35.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:35.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:35.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:35.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:35.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:35.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:35.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-24T11:27:35.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: osdmap e131: 8 total, 7 up, 8 in 2024-03-24T11:27:35.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: from='osd.2 [v2:172.21.15.62:6818/904171279,v1:172.21.15.62:6819/904171279]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:27:35.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:27:35.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:35.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:35.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:35.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:35.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:35.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:35.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:35.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:35.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:35.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:35.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:35.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:35.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:35.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:35.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:35.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: pgmap v143: 161 pgs: 14 active+undersized+degraded, 32 active+undersized, 115 active+clean; 585 KiB data, 154 MiB used, 715 GiB / 715 GiB avail; 56/630 objects degraded (8.889%) 2024-03-24T11:27:35.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:35.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:35.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:35.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:35.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:35.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:35.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-24T11:27:35.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: osdmap e131: 8 total, 7 up, 8 in 2024-03-24T11:27:35.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: from='osd.2 [v2:172.21.15.62:6818/904171279,v1:172.21.15.62:6819/904171279]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:27:35.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:27:35.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:35.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:35.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:35.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:35.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:35.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:35.500 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:35.500 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:35.500 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:35.500 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:35.500 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:35.500 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:35.500 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:35.500 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: Health check update: Degraded data redundancy: 56/630 objects degraded (8.889%), 14 pgs degraded (PG_DEGRADED) 2024-03-24T11:27:35.500 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:35.500 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:35.500 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:35.500 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:35.501 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:35.501 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:35.501 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:35.501 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:35.501 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:35.501 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:35.501 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:35.501 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:35.501 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:35.501 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:35.501 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:35.501 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:35.501 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:35.501 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:35.502 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:35.502 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:35.502 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:35.502 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:35.502 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:35.502 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:35.502 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:35.502 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:35.502 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:35.502 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:35.502 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:35.502 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:35.502 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: pgmap v143: 161 pgs: 14 active+undersized+degraded, 32 active+undersized, 115 active+clean; 585 KiB data, 154 MiB used, 715 GiB / 715 GiB avail; 56/630 objects degraded (8.889%) 2024-03-24T11:27:35.503 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:35.503 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:35.503 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:35.503 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:35.503 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:35.503 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:35.503 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-24T11:27:35.503 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: osdmap e131: 8 total, 7 up, 8 in 2024-03-24T11:27:35.503 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: from='osd.2 [v2:172.21.15.62:6818/904171279,v1:172.21.15.62:6819/904171279]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:27:35.503 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:27:35.503 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:35.503 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:35.503 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:35.503 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:35.503 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:35.504 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:35 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:36.460 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: failed to encode map e131 with expected crc 2024-03-24T11:27:36.460 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: failed to encode map e131 with expected crc 2024-03-24T11:27:36.460 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: failed to encode map e131 with expected crc 2024-03-24T11:27:36.461 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: failed to encode map e131 with expected crc 2024-03-24T11:27:36.461 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:36.461 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:36.461 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:36.461 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:36.461 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:36.461 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:36.461 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: from='osd.2 ' entity='osd.2' 2024-03-24T11:27:36.461 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:36.461 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:36.461 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:36.461 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:36.461 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:36.461 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:36.462 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:36.462 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:36.462 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:36.462 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:36.462 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:36.462 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:36.462 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:36.462 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:36.462 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:36.462 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:36.462 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:36.462 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:36.463 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:36.463 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:36.463 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:36.463 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:36.463 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:36.463 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:36.463 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:36.463 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:36.463 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:36.463 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:36.463 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:36.463 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:36.463 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: failed to encode map e131 with expected crc 2024-03-24T11:27:36.463 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:36.464 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:36.464 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:36.464 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:36.464 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:36.464 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:36.464 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:36.464 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:36.464 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:36.464 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:36.464 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:36.464 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:36.464 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:36.464 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:36.464 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:36.465 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:36.465 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:36.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: failed to encode map e131 with expected crc 2024-03-24T11:27:36.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: failed to encode map e131 with expected crc 2024-03-24T11:27:36.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: failed to encode map e131 with expected crc 2024-03-24T11:27:36.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: failed to encode map e131 with expected crc 2024-03-24T11:27:36.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:36.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:36.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:36.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:36.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:36.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:36.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: from='osd.2 ' entity='osd.2' 2024-03-24T11:27:36.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:36.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:36.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:36.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:36.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:36.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:36.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:36.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:36.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:36.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:36.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:36.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:36.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:36.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:36.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:36.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:36.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:36.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:36.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:36.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:36.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: failed to encode map e131 with expected crc 2024-03-24T11:27:36.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: failed to encode map e131 with expected crc 2024-03-24T11:27:36.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: failed to encode map e131 with expected crc 2024-03-24T11:27:36.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: failed to encode map e131 with expected crc 2024-03-24T11:27:36.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:36.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:36.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:36.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:36.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:36.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:36.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: from='osd.2 ' entity='osd.2' 2024-03-24T11:27:36.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:36.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:36.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:36.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:36.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:36.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:36.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:36.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:36.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:36.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:36.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:36.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:36.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:36.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:36.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:36.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:36.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:36.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:36.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:36.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:36.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:36.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:36.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:36.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:36.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:36.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:36.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:36.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:36.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:36.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:36.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: failed to encode map e131 with expected crc 2024-03-24T11:27:36.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:36.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:36.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:36.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:36.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:36.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:36.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:36.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:36.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:36.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:36.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:36.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:36.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:36.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:36.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:36.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:36.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:36.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:36.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:36.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:36.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:36.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:36.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:36.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:36.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:36.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:36.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:36.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: failed to encode map e131 with expected crc 2024-03-24T11:27:36.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:36.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:36.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:36.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:36.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:36.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:36.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:36.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:36.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:36.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:36.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:36.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:36.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:36.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:36.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:36.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:36.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:37.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:37.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:37.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:37.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:37.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:37.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:37.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:37.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T11:27:37.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: failed to encode map e132 with expected crc 2024-03-24T11:27:37.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: osd.2 [v2:172.21.15.62:6818/904171279,v1:172.21.15.62:6819/904171279] boot 2024-03-24T11:27:37.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: osdmap e132: 8 total, 8 up, 8 in 2024-03-24T11:27:37.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T11:27:37.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:37.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:37.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:37.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:37.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:37.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:37.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:37.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:37.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:37.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:37.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:37.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:37.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:37.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:37.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:37.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:37.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:37.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:37.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:37.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:37.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:37.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:37.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:37.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:37.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:37.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:37.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:37.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:37.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:37.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:37.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: pgmap v146: 161 pgs: 14 active+undersized+degraded, 32 active+undersized, 115 active+clean; 585 KiB data, 169 MiB used, 715 GiB / 715 GiB avail; 762 B/s rd, 0 op/s; 56/630 objects degraded (8.889%) 2024-03-24T11:27:37.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:37.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:37.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:37.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:37.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:37.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:37.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:37.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:37.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:37.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:37.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:37.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:37.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:37.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:37.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:37.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:37.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:37.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:37.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:37.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:37.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:37.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:37.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:37.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:37.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:37.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:37.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:37.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:37.364 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:37.394 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:37.394 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:37.394 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:37.394 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:37.394 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:37.395 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:37.395 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:37.395 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T11:27:37.395 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: failed to encode map e132 with expected crc 2024-03-24T11:27:37.395 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: osd.2 [v2:172.21.15.62:6818/904171279,v1:172.21.15.62:6819/904171279] boot 2024-03-24T11:27:37.395 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: osdmap e132: 8 total, 8 up, 8 in 2024-03-24T11:27:37.395 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T11:27:37.395 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:37.395 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:37.395 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:37.395 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:37.395 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:37.395 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:37.395 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:37.395 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:37.396 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:37.396 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:37.396 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:37.396 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:37.396 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:37.396 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:37.396 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:37.396 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:37.396 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:37.396 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:37.396 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:37.397 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:37.397 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:37.397 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:37.397 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:37.397 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:37.397 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:37.397 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T11:27:37.397 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: failed to encode map e132 with expected crc 2024-03-24T11:27:37.397 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: osd.2 [v2:172.21.15.62:6818/904171279,v1:172.21.15.62:6819/904171279] boot 2024-03-24T11:27:37.397 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: osdmap e132: 8 total, 8 up, 8 in 2024-03-24T11:27:37.397 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T11:27:37.397 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:37.397 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:37.397 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:37.397 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:37.398 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:37.398 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:37.398 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:37.398 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:37.398 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:37.398 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:37.398 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:37.398 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:37.398 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:37.398 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:37.398 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:37.398 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:37.398 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:37.398 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:37.399 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:37.399 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:37.399 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:37.399 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:37.399 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:37.399 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:37.399 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:37.400 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:37.400 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:37.400 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:37.400 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:37.400 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:37.400 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: pgmap v146: 161 pgs: 14 active+undersized+degraded, 32 active+undersized, 115 active+clean; 585 KiB data, 169 MiB used, 715 GiB / 715 GiB avail; 762 B/s rd, 0 op/s; 56/630 objects degraded (8.889%) 2024-03-24T11:27:37.400 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:37.400 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:37.400 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:37.400 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:37.400 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:37.401 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:37.401 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:37.401 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:37.401 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:37.401 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:37.401 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:37.401 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:37.401 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:37.401 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:37.401 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:37.401 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:37.402 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:37.402 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:37.402 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:37.402 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:37.402 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:37.402 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:37.402 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:37.402 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:37.402 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:37.402 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:37.402 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:37.402 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:37.402 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: pgmap v146: 161 pgs: 14 active+undersized+degraded, 32 active+undersized, 115 active+clean; 585 KiB data, 169 MiB used, 715 GiB / 715 GiB avail; 762 B/s rd, 0 op/s; 56/630 objects degraded (8.889%) 2024-03-24T11:27:37.402 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:37.403 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:37.403 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:37.403 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:37.403 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:37.403 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:37.403 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:37.403 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:37.403 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:37.403 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:37.403 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:37.403 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:37.403 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:37.403 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:37.404 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:37.404 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:37.404 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:37.404 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:37.404 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:37.404 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:37.404 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:37.404 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:37.404 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:37.404 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:37.405 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:37.405 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:37.405 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:37.405 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:37.405 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:37.405 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:37.405 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:37.405 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:37.405 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:37.405 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:37.405 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:37.405 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:37.406 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:37.406 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:37.406 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:37.406 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:37.406 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:37.406 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:38.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: failed to encode map e132 with expected crc 2024-03-24T11:27:38.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:38.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:38.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:38.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:38.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:38.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:38.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:38.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: failed to encode map e133 with expected crc 2024-03-24T11:27:38.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: osdmap e133: 8 total, 8 up, 8 in 2024-03-24T11:27:38.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: failed to encode map e133 with expected crc 2024-03-24T11:27:38.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: failed to encode map e133 with expected crc 2024-03-24T11:27:38.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: failed to encode map e133 with expected crc 2024-03-24T11:27:38.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: failed to encode map e133 with expected crc 2024-03-24T11:27:38.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: failed to encode map e133 with expected crc 2024-03-24T11:27:38.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: failed to encode map e133 with expected crc 2024-03-24T11:27:38.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: failed to encode map e133 with expected crc 2024-03-24T11:27:38.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: failed to encode map e133 with expected crc 2024-03-24T11:27:38.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:38.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:38.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:38.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:38.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:38.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:38.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:38.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:38.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:38.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:38.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:38.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:38.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:38.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:38.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:38.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: failed to encode map e132 with expected crc 2024-03-24T11:27:38.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:38.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:38.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:38.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:38.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:38.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:38.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:38.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: failed to encode map e133 with expected crc 2024-03-24T11:27:38.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: osdmap e133: 8 total, 8 up, 8 in 2024-03-24T11:27:38.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: failed to encode map e133 with expected crc 2024-03-24T11:27:38.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: failed to encode map e133 with expected crc 2024-03-24T11:27:38.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: failed to encode map e133 with expected crc 2024-03-24T11:27:38.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: failed to encode map e133 with expected crc 2024-03-24T11:27:38.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: failed to encode map e133 with expected crc 2024-03-24T11:27:38.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: failed to encode map e133 with expected crc 2024-03-24T11:27:38.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: failed to encode map e133 with expected crc 2024-03-24T11:27:38.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: failed to encode map e133 with expected crc 2024-03-24T11:27:38.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:38.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:38.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:38.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:38.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:38.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:38.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:38.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:38.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:38.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:38.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:38.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:38.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:38.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:38.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:38.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:38.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:38.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:38.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:38.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:38.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:38.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:38.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:38.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:38.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:38.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:38.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:38.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:38.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:38.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:38.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:38.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:38.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:38.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:38.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:38.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:38.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:38.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:38.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:38.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:38.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:38.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:38.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:38.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:38.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:38.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:38.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:38.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:38.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:38.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:38.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:38.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:38.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:38.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:38.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:38.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:38.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:38.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:38.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:38.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:38.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:38.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:38.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:38.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:38.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:38.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:38.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:38.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:38.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:38.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:38.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:38.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:38.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:38.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:38.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:38.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:38.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:38.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:38.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:38.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:38.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:38.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:38.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:38.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:38.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:38.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:38.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:38.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:38.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:38.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:38.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:38.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:38.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:38.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:38.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:38.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:38.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:38.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:38.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:38.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:38.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:38.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:38.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:38.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: failed to encode map e132 with expected crc 2024-03-24T11:27:38.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:38.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:38.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:38.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:38.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:38.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:38.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:38.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: failed to encode map e133 with expected crc 2024-03-24T11:27:38.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: osdmap e133: 8 total, 8 up, 8 in 2024-03-24T11:27:38.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: failed to encode map e133 with expected crc 2024-03-24T11:27:38.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: failed to encode map e133 with expected crc 2024-03-24T11:27:38.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: failed to encode map e133 with expected crc 2024-03-24T11:27:38.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: failed to encode map e133 with expected crc 2024-03-24T11:27:38.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: failed to encode map e133 with expected crc 2024-03-24T11:27:38.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: failed to encode map e133 with expected crc 2024-03-24T11:27:38.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: failed to encode map e133 with expected crc 2024-03-24T11:27:38.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: failed to encode map e133 with expected crc 2024-03-24T11:27:38.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:38.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:38.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:38.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:38.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:38.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:38.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:38.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:38.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:38.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:38.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:38.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:38.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:38.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:38.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:38.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:38.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:38.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:38.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:38.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:38.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:38.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:38.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:38.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:38.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:38.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:38.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:38.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:38.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:38.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:38.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:38.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:38.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:38.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:38.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:38.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:38.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:38.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:38.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:38.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:38.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:38.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:38.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:38.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:38.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:38.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:38.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:38.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:38.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:38.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:38.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:38.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:38.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:38.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:38.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:38.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:38.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:38.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:38.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:39.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:39.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:39.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:39.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:39.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:39.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:39.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:39.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:39.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:39.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:39.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:39.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:39.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:39.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:39.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:39.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:39.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:39.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:39.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:39.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:39.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:39.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:39.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:39.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:39.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:39.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:39.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:39.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:39.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:39.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:39.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:39.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:39.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:39.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:39.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:39.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:39.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:39.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:39.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:39.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:39.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:39.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:39.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:39.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:39.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:39.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:39.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:39.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:39.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:39.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:39.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:39.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:39.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:39.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:39.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:39.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:39.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:39.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:39.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:39.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:39.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: pgmap v148: 161 pgs: 23 peering, 3 active+undersized+degraded, 15 active+undersized, 120 active+clean; 585 KiB data, 173 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 7/630 objects degraded (1.111%) 2024-03-24T11:27:39.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:39.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:39.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:39.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:39.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:39.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:39.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:39.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:39.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:39.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:39.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:39.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:39.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:39.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:39.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:39.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:39.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:39.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:39.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:39.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:39.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:39.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:39.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:39.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:39.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:39.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: pgmap v148: 161 pgs: 23 peering, 3 active+undersized+degraded, 15 active+undersized, 120 active+clean; 585 KiB data, 173 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 7/630 objects degraded (1.111%) 2024-03-24T11:27:39.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:39.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:39.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:39.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:39.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:39.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:39.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:39.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:39.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:39.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:39.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:39.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:39.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:39.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:39.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:39.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:39.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:39.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:39.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:39.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:39.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:39.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:39.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:39.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:39.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:39.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:39.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:39.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:39.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:39.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:39.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:39.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:39.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:39.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:39.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:39.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:39.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:39.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:39.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:39.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:39.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:39.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:39.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:39.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:39.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:39.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:39.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:39.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:39.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:39.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:39.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:39.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:39.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:39.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:39.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: pgmap v148: 161 pgs: 23 peering, 3 active+undersized+degraded, 15 active+undersized, 120 active+clean; 585 KiB data, 173 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 7/630 objects degraded (1.111%) 2024-03-24T11:27:39.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:39.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:39.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:39.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:39.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:39.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:39.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:39.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:39.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:39.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:39.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:39.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:39.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:39.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:39.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:39.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:39.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:39.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:39.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:39.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:39.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:39.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:39.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:40.385 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:40.385 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:40.385 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:40.385 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:40.385 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:40.386 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:40.386 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:40.386 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:40.386 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:40.386 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:40.386 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:40.386 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:40.386 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:40.386 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:40.386 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:40.386 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:40.386 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:40.386 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:40.387 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:40.387 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:40.387 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:40.387 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:40.387 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:40.387 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:40.387 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:40.387 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: Health check update: Degraded data redundancy: 7/630 objects degraded (1.111%), 3 pgs degraded (PG_DEGRADED) 2024-03-24T11:27:40.387 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:40.387 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:40.387 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:40.387 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:40.388 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:40.388 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:40.388 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:40.388 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:40.388 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:40.388 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:40.388 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:40.388 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:40.388 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:40.388 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:40.388 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:40.388 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:40.389 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:40.389 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:40.389 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:40.389 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:40.389 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:40.389 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:40.389 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:40.389 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:40.389 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:40.389 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:40.389 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:40.389 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:40.389 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:40.389 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: Health check update: Degraded data redundancy: 7/630 objects degraded (1.111%), 3 pgs degraded (PG_DEGRADED) 2024-03-24T11:27:40.390 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:40.390 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:40.390 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:40.390 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:40.390 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:40.390 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:40.390 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:40.390 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:40.390 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:40.391 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:40.391 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:40.391 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:40.391 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:40.391 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:40.391 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:40.391 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:40.391 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:40.391 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:40.391 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:40.391 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:40.391 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:40.391 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:40.391 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:40.392 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:40.392 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:40.392 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:40.392 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:40.392 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:40.392 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:40.392 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:40.392 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:40.392 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:40.392 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:40.392 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:40.392 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:40.392 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:40.392 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:40.393 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:40.393 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:40.393 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:40.393 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:40.393 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:40.393 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:40.393 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:40.393 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:40.393 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:40.393 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:40.394 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:40.394 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:40.394 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:40.394 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:40.394 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:40.394 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:40.394 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:40.394 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:40.394 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:40.394 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:40.395 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:40.395 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:40.395 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:40.395 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:40.395 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:40.395 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:40.395 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:40.395 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:40.395 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:40.395 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:40.395 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:40.395 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:40.396 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:40.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:40.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:40.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:40.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:40.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:40.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:40.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:40.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:40.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:40.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:40.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:40.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:40.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:40.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:40.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:40.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:40.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:40.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:40.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:40.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:40.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:40.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:40.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:40.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:40.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:40.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: Health check update: Degraded data redundancy: 7/630 objects degraded (1.111%), 3 pgs degraded (PG_DEGRADED) 2024-03-24T11:27:40.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:40.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:40.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:40.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:40.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:40.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:40.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:40.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:40.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:40.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:40.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:40.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:40.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:40.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:40.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:40.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:40.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:40.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:40.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:40.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:40.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:40.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:40.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:40.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:40.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:40.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:40.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:40.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:40.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:40.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:40.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:40.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:40.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:40.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:40.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:40.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:40.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:41.145 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:27:40 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:27:40] "GET /metrics HTTP/1.1" 200 38501 "" "Prometheus/2.33.4" 2024-03-24T11:27:41.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:41.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:41.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:41.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:41.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:41.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:41.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:41.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:41.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:41.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:41.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:41.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:41.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:41.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:41.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:41.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:41.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:41.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:41.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:41.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:41.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:41.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:41.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:41.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:41.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:41.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:41.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: pgmap v149: 161 pgs: 23 peering, 138 active+clean; 585 KiB data, 173 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:27:41.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:41.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:41.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:41.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:41.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:41.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:41.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:41.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:41.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:41.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:41.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:41.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:41.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:41.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:41.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:41.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:41.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:41.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:41.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:41.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:41.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:41.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:41.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:41.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:41.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:41.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:41.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:41.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:41.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: pgmap v149: 161 pgs: 23 peering, 138 active+clean; 585 KiB data, 173 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:27:41.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:41.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:41.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:41.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:41.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:41.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:41.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:41.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:41.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:41.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:41.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:41.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:41.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:41.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:41.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:41.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:41.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:41.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:41.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:41.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:41.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:41.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:41.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:41.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:41.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:41.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:41.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:41.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:41.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:41.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:41.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:41.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:41.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:41.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:41.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:41.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:41.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:41.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:41.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:41.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:41.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:41.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:41.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:41.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:41.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:41.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:41.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:41.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:41.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:41.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:41.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:41.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:41.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:41.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:41.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:41.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:41.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:41.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:41.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:41.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:41.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:41.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:41.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:41.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:41.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:41.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:41.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:41.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:41.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:41.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:41.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:41.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:41.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:41.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:41.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:41.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:41.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:41.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:41.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:41.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:41.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:41.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:41.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:41.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:41.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:41.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:41.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:41.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:41.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:41.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:41.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:41.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:41.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:41.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:41.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: pgmap v149: 161 pgs: 23 peering, 138 active+clean; 585 KiB data, 173 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:27:41.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:41.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:41.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:41.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:41.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:41.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:41.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:41.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:41.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:41.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:41.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:41.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:41.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:41.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:41.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:41.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:41.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:41.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:41.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:41.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:41.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:41.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:41.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:41.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:41.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:41.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:41.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:41.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:41.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:41.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:41.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:41.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:41.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:41.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:41.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:42.424 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:42.425 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:42.425 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:27:42.425 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:42.425 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:42.425 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:42.425 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:42.425 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:42.425 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:42.425 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:42.425 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:42.425 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:42.425 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:42.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:42.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:42.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:42.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:42.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:42.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:42.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:42.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:42.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:42.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:42.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:42.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:42.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:42.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:42.427 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 7/630 objects degraded (1.111%), 3 pgs degraded) 2024-03-24T11:27:42.427 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: Cluster is now healthy 2024-03-24T11:27:42.427 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:42.427 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:42.427 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:27:42.427 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:42.427 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:42.427 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:42.427 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:42.427 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:42.427 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:42.428 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:42.428 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:42.428 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:42.428 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:42.428 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:42.428 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:42.428 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:42.428 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:42.428 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:42.428 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:42.428 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:42.428 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:42.428 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:42.428 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:42.428 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:42.428 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:42.428 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:42.429 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:42.429 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 7/630 objects degraded (1.111%), 3 pgs degraded) 2024-03-24T11:27:42.429 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: Cluster is now healthy 2024-03-24T11:27:42.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:42.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:42.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:42.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:42.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:42.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:42.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:42.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:42.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:42.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:42.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:42.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:42.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:42.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:42.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:42.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:42.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:42.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:42.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:42.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:42.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:42.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:42.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:42.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:42.431 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:42.431 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:42.431 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:42.431 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:42.431 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:42.431 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:42.431 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:42.431 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:42.431 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:42.431 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:42.431 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:42.431 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:42.431 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:42.431 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:42.432 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:42.432 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:42.432 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:42.432 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:42.432 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:42.432 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:42.432 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:42.432 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:42.432 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:42.432 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:42.432 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:42.432 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:42.432 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:42.432 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:42.432 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:42.433 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:42.433 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:42.433 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:42.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:42.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:42.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:27:42.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:42.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:42.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:42.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:42.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:42.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:42.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:42.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:42.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:42.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:42.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:42.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:42.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:42.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:42.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:42.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:42.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:42.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:42.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:42.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:42.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:42.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:42.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:42.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:42.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 7/630 objects degraded (1.111%), 3 pgs degraded) 2024-03-24T11:27:42.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: Cluster is now healthy 2024-03-24T11:27:42.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:42.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:42.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:42.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:42.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:42.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:42.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:42.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:42.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:42.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:42.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:42.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:42.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:42.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:42.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:42.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:42.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:42.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:42.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:42.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:42.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:42.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:42.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:42.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:42.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:42.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:42.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:42.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:43.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:43.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:43.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:27:43.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:43.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:43.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:43.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:43.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:43.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:43.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:43.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:43.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:43.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:43.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:43.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:43.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:43.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:43.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:43.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:43.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:43.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:43.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:43.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:43.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:43.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:43.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:43.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:43.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: pgmap v150: 161 pgs: 161 active+clean; 585 KiB data, 173 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T11:27:43.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:43.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:43.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:43.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:27:43.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:43.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:43.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:43.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:43.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:43.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:43.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:43.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:43.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:43.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:43.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:43.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:43.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:43.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:43.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:43.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:43.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:43.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:43.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:43.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:43.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:43.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:43.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:43.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:43.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: pgmap v150: 161 pgs: 161 active+clean; 585 KiB data, 173 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T11:27:43.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:43.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:43.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:43.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:43.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:43.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:43.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:43.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:43.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:43.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:43.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:43.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:43.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:43.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:43.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:43.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:43.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:43.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:43.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:43.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:43.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:43.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:43.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:43.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:43.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:43.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:43.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:43.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:43.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:43.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:43.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:43.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:43.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:43.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:43.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:43.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:43.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:43.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:43.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:43.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:43.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:43.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:43.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:43.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:43.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:43.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:43.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:43.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:43.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:43.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:43.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:43.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:43.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:43.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:43.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:43.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:43.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:43.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:43.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:43.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:43.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:43.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:43.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:43.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:43.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:43.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:43.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:43.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:43.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:43.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:43.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:43.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:27:43.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:43.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:43.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:43.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:43.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:43.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:43.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:43.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:43.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:43.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:43.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:43.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:43.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:43.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:43.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:43.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:43.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:43.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:43.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:43.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:43.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:43.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:43.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:43.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:43.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: pgmap v150: 161 pgs: 161 active+clean; 585 KiB data, 173 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T11:27:43.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:43.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:43.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:43.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:43.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:43.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:43.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:43.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:43.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:43.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:43.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:43.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:43.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:43.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:43.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:43.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:43.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:43.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:43.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:43.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:43.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:43.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:43.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:43.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:43.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:43.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:43.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:43.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:43.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:43.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:43.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:43.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:43.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:43.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:43.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:44.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:44.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:44.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:44.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:44.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:44.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:44.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:44.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:44.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:44.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:44.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:44.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:44.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:44.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:44.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:44.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:44.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:44.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:44.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:44.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:44.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:44.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:44.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:44.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:44.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:44.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:44.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:44.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:44.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:44.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:44.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:44.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:44.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:44.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:44.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:44.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:44.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:44.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:44.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:44.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:44.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:44.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:44.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:44.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:44.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:44.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:44.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:44.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:44.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:44.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:44.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:44.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:44.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:44.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:44.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:44.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:44.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:44.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:44.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:44.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:44.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:44.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:44.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:44.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:44.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:44.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:44.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:44.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:44.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:44.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:44.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:44.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:44.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:44.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:44.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:44.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:44.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:44.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:44.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:44.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:44.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:44.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:44.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:44.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:44.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:44.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:44.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:44.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:44.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:44.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:44.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:44.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:44.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:44.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:44.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:44.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:44.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:44.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:44.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:44.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:44.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:44.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:44.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-24T11:27:44.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:44.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:44.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:44.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:44.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:44.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:44.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:44.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:44.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:44.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:44.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:44.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:44.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:44.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:44.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:44.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:44.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:44.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:44.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:44.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:44.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:44.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:44.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:44.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:44.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:44.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:44.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:44.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:44.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:44.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:44.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:44.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:44.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:44.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:44.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:44.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:44.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:44.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:44.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:44.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:44.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:44.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:44.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:44.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:44.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-24T11:27:44.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:44.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:44.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:44.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:44.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:44.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:44.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:44.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:44.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:44.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:44.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:44.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:44.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:44.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:44.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:44.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:44.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:44.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:44.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:44.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:44.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:44.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:44.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:44.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:44.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:44.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:44.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:44.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:44.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:44.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:44.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:44.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:44.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:44.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:44.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:44.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:44.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:44.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:44.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:44.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:44.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:44.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:44.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:44.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:44.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:44.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:44.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:44.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:44.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:44.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:44.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:44.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:44.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:44.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:44.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:44.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:44.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:44.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:44.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:44.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:44.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:44.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:44.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:44.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:44.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:44.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:44.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:44.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:44.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:44.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:44.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:44.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:44.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:44.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-24T11:27:45.306 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:45 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:45.306 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:45 smithi062 ceph-mon[98643]: pgmap v151: 161 pgs: 161 active+clean; 585 KiB data, 173 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T11:27:45.306 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:45 smithi062 ceph-mon[98643]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-24T11:27:45.306 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:45 smithi062 ceph-mon[98643]: Upgrade: osd.2 is safe to restart 2024-03-24T11:27:45.306 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:45 smithi062 ceph-mon[98643]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-03-24T11:27:45.307 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:45 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:45.307 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:45 smithi062 ceph-mon[90553]: pgmap v151: 161 pgs: 161 active+clean; 585 KiB data, 173 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T11:27:45.307 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:45 smithi062 ceph-mon[90553]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-24T11:27:45.307 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:45 smithi062 ceph-mon[90553]: Upgrade: osd.2 is safe to restart 2024-03-24T11:27:45.307 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:45 smithi062 ceph-mon[90553]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-03-24T11:27:45.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:45 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:45.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:45 smithi139 ceph-mon[91894]: pgmap v151: 161 pgs: 161 active+clean; 585 KiB data, 173 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T11:27:45.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:45 smithi139 ceph-mon[91894]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-24T11:27:45.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:45 smithi139 ceph-mon[91894]: Upgrade: osd.2 is safe to restart 2024-03-24T11:27:45.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:45 smithi139 ceph-mon[91894]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-03-24T11:27:47.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:47 smithi062 ceph-mon[98643]: pgmap v152: 161 pgs: 161 active+clean; 585 KiB data, 173 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-24T11:27:47.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:47 smithi062 ceph-mon[98643]: Upgrade: Updating osd.2 2024-03-24T11:27:47.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-24T11:27:47.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:47.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:47 smithi062 ceph-mon[98643]: Deploying daemon osd.2 on smithi062 2024-03-24T11:27:47.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:47.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:47.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:47.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:47 smithi062 ceph-mon[90553]: pgmap v152: 161 pgs: 161 active+clean; 585 KiB data, 173 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-24T11:27:47.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:47 smithi062 ceph-mon[90553]: Upgrade: Updating osd.2 2024-03-24T11:27:47.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-24T11:27:47.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:47.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:47 smithi062 ceph-mon[90553]: Deploying daemon osd.2 on smithi062 2024-03-24T11:27:47.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:47.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:47.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:47.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:47 smithi139 ceph-mon[91894]: pgmap v152: 161 pgs: 161 active+clean; 585 KiB data, 173 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-24T11:27:47.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:47 smithi139 ceph-mon[91894]: Upgrade: Updating osd.2 2024-03-24T11:27:47.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-24T11:27:47.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:47.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:47 smithi139 ceph-mon[91894]: Deploying daemon osd.2 on smithi062 2024-03-24T11:27:47.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:47.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:47.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:48.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:48 smithi062 ceph-mon[98643]: pgmap v153: 161 pgs: 161 active+clean; 585 KiB data, 173 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-24T11:27:48.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:48 smithi062 ceph-mon[98643]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-03-24T11:27:48.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:48 smithi062 ceph-mon[98643]: Cluster is now healthy 2024-03-24T11:27:48.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:48 smithi062 ceph-mon[90553]: pgmap v153: 161 pgs: 161 active+clean; 585 KiB data, 173 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-24T11:27:48.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:48 smithi062 ceph-mon[90553]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-03-24T11:27:48.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:48 smithi062 ceph-mon[90553]: Cluster is now healthy 2024-03-24T11:27:48.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:48 smithi139 ceph-mon[91894]: pgmap v153: 161 pgs: 161 active+clean; 585 KiB data, 173 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-24T11:27:48.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:48 smithi139 ceph-mon[91894]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-03-24T11:27:48.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:48 smithi139 ceph-mon[91894]: Cluster is now healthy 2024-03-24T11:27:50.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:50 smithi062 ceph-mon[90553]: pgmap v154: 161 pgs: 161 active+clean; 585 KiB data, 173 MiB used, 715 GiB / 715 GiB avail; 979 B/s rd, 0 op/s 2024-03-24T11:27:50.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:50 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:50.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:50 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:50.370 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:50 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:50.371 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:50 smithi062 ceph-mon[98643]: pgmap v154: 161 pgs: 161 active+clean; 585 KiB data, 173 MiB used, 715 GiB / 715 GiB avail; 979 B/s rd, 0 op/s 2024-03-24T11:27:50.371 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:50 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:50.371 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:50 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:50.371 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:50 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:50.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:50 smithi139 ceph-mon[91894]: pgmap v154: 161 pgs: 161 active+clean; 585 KiB data, 173 MiB used, 715 GiB / 715 GiB avail; 979 B/s rd, 0 op/s 2024-03-24T11:27:50.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:50 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:50.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:50 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:50.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:50 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:50.934 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:27:50 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:27:50] "GET /metrics HTTP/1.1" 200 38510 "" "Prometheus/2.33.4" 2024-03-24T11:27:51.977 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:51 smithi062 systemd[1]: Stopping Ceph osd.2 for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:27:52.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:52 smithi062 ceph-mon[98643]: pgmap v155: 161 pgs: 161 active+clean; 585 KiB data, 173 MiB used, 715 GiB / 715 GiB avail; 979 B/s rd, 0 op/s 2024-03-24T11:27:52.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:52 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:52.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:52 smithi062 ceph-mon[98643]: osd.2 marked itself down and dead 2024-03-24T11:27:52.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:52 smithi062 ceph-mon[90553]: pgmap v155: 161 pgs: 161 active+clean; 585 KiB data, 173 MiB used, 715 GiB / 715 GiB avail; 979 B/s rd, 0 op/s 2024-03-24T11:27:52.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:52 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:52.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:52 smithi062 ceph-mon[90553]: osd.2 marked itself down and dead 2024-03-24T11:27:52.478 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:52 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2[130691]: 2024-03-24T11:27:52.159+0000 7f0bb999a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T11:27:52.478 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:52 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2[130691]: 2024-03-24T11:27:52.159+0000 7f0bb999a700 -1 osd.2 133 *** Got signal Terminated *** 2024-03-24T11:27:52.478 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:52 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2[130691]: 2024-03-24T11:27:52.159+0000 7f0bb999a700 -1 osd.2 133 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T11:27:52.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:52 smithi139 ceph-mon[91894]: pgmap v155: 161 pgs: 161 active+clean; 585 KiB data, 173 MiB used, 715 GiB / 715 GiB avail; 979 B/s rd, 0 op/s 2024-03-24T11:27:52.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:52 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:52.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:52 smithi139 ceph-mon[91894]: osd.2 marked itself down and dead 2024-03-24T11:27:52.852 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:52 smithi062 podman[137032]: 2024-03-24 11:27:52.57892791 +0000 UTC m=+0.529172970 container died da9e6fa160245eef870511413321664d0b1aaa4d125f560a52aeea31ac984100 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-24T11:27:53.227 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:52 smithi062 podman[137032]: 2024-03-24 11:27:52.951483542 +0000 UTC m=+0.901728597 container cleanup da9e6fa160245eef870511413321664d0b1aaa4d125f560a52aeea31ac984100 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, ceph=True) 2024-03-24T11:27:53.227 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:52 smithi062 bash[137032]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2 2024-03-24T11:27:53.572 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:53 smithi062 podman[137049]: 2024-03-24 11:27:53.228040916 +0000 UTC m=+0.643056332 container remove da9e6fa160245eef870511413321664d0b1aaa4d125f560a52aeea31ac984100 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-db0330b, ceph=True, CEPH_POINT_RELEASE=) 2024-03-24T11:27:53.572 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:53 smithi062 ceph-mon[98643]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T11:27:53.572 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:53 smithi062 ceph-mon[98643]: osdmap e134: 8 total, 7 up, 8 in 2024-03-24T11:27:53.572 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:53 smithi062 ceph-mon[98643]: failed to encode map e134 with expected crc 2024-03-24T11:27:53.573 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:53 smithi062 ceph-mon[98643]: failed to encode map e134 with expected crc 2024-03-24T11:27:53.573 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:53 smithi062 ceph-mon[98643]: failed to encode map e134 with expected crc 2024-03-24T11:27:53.573 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:53 smithi062 ceph-mon[98643]: failed to encode map e134 with expected crc 2024-03-24T11:27:53.573 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:53 smithi062 ceph-mon[98643]: failed to encode map e134 with expected crc 2024-03-24T11:27:53.573 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:53 smithi062 ceph-mon[98643]: failed to encode map e134 with expected crc 2024-03-24T11:27:53.573 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:53 smithi062 ceph-mon[98643]: failed to encode map e134 with expected crc 2024-03-24T11:27:53.573 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:53 smithi062 ceph-mon[98643]: failed to encode map e134 with expected crc 2024-03-24T11:27:53.573 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:53 smithi062 ceph-mon[98643]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:27:53.573 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:53 smithi062 ceph-mon[90553]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T11:27:53.573 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:53 smithi062 ceph-mon[90553]: osdmap e134: 8 total, 7 up, 8 in 2024-03-24T11:27:53.573 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:53 smithi062 ceph-mon[90553]: failed to encode map e134 with expected crc 2024-03-24T11:27:53.573 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:53 smithi062 ceph-mon[90553]: failed to encode map e134 with expected crc 2024-03-24T11:27:53.574 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:53 smithi062 ceph-mon[90553]: failed to encode map e134 with expected crc 2024-03-24T11:27:53.574 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:53 smithi062 ceph-mon[90553]: failed to encode map e134 with expected crc 2024-03-24T11:27:53.574 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:53 smithi062 ceph-mon[90553]: failed to encode map e134 with expected crc 2024-03-24T11:27:53.574 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:53 smithi062 ceph-mon[90553]: failed to encode map e134 with expected crc 2024-03-24T11:27:53.574 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:53 smithi062 ceph-mon[90553]: failed to encode map e134 with expected crc 2024-03-24T11:27:53.574 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:53 smithi062 ceph-mon[90553]: failed to encode map e134 with expected crc 2024-03-24T11:27:53.574 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:53 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:27:53.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:53 smithi139 ceph-mon[91894]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T11:27:53.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:53 smithi139 ceph-mon[91894]: osdmap e134: 8 total, 7 up, 8 in 2024-03-24T11:27:53.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:53 smithi139 ceph-mon[91894]: failed to encode map e134 with expected crc 2024-03-24T11:27:53.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:53 smithi139 ceph-mon[91894]: failed to encode map e134 with expected crc 2024-03-24T11:27:53.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:53 smithi139 ceph-mon[91894]: failed to encode map e134 with expected crc 2024-03-24T11:27:53.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:53 smithi139 ceph-mon[91894]: failed to encode map e134 with expected crc 2024-03-24T11:27:53.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:53 smithi139 ceph-mon[91894]: failed to encode map e134 with expected crc 2024-03-24T11:27:53.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:53 smithi139 ceph-mon[91894]: failed to encode map e134 with expected crc 2024-03-24T11:27:53.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:53 smithi139 ceph-mon[91894]: failed to encode map e134 with expected crc 2024-03-24T11:27:53.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:53 smithi139 ceph-mon[91894]: failed to encode map e134 with expected crc 2024-03-24T11:27:53.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:53 smithi139 ceph-mon[91894]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:27:53.865 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:53 smithi062 podman[137137]: 2024-03-24 11:27:53.47505546 +0000 UTC m=+0.017522380 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:27:53.865 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:53 smithi062 podman[137137]: 2024-03-24 11:27:53.579142058 +0000 UTC m=+0.121608962 container create af198930cef743c48b960e06a1ba1de7d2b3e4f68efbea05633b6886f2545c52 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-24T11:27:53.865 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:53 smithi062 podman[137137]: 2024-03-24 11:27:53.782566099 +0000 UTC m=+0.325033019 container init af198930cef743c48b960e06a1ba1de7d2b3e4f68efbea05633b6886f2545c52 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2-deactivate, CEPH_POINT_RELEASE=, RELEASE=squid-db0330b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-24T11:27:53.865 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:53 smithi062 podman[137137]: 2024-03-24 11:27:53.786055752 +0000 UTC m=+0.328522659 container start af198930cef743c48b960e06a1ba1de7d2b3e4f68efbea05633b6886f2545c52 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, ceph=True) 2024-03-24T11:27:53.866 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:53 smithi062 podman[137137]: 2024-03-24 11:27:53.865313554 +0000 UTC m=+0.407780468 container attach af198930cef743c48b960e06a1ba1de7d2b3e4f68efbea05633b6886f2545c52 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2-deactivate, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-24T11:27:54.119 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:54 smithi062 podman[137137]: 2024-03-24 11:27:54.093003756 +0000 UTC m=+0.635470670 container died af198930cef743c48b960e06a1ba1de7d2b3e4f68efbea05633b6886f2545c52 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2-deactivate, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-24T11:27:54.444 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:54 smithi062 ceph-mon[90553]: pgmap v157: 161 pgs: 13 stale+active+clean, 148 active+clean; 585 KiB data, 173 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-24T11:27:54.444 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:54 smithi062 ceph-mon[90553]: failed to encode map e135 with expected crc 2024-03-24T11:27:54.445 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:54 smithi062 ceph-mon[90553]: failed to encode map e135 with expected crc 2024-03-24T11:27:54.445 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:54 smithi062 ceph-mon[90553]: osdmap e135: 8 total, 7 up, 8 in 2024-03-24T11:27:54.445 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:54 smithi062 ceph-mon[90553]: failed to encode map e135 with expected crc 2024-03-24T11:27:54.445 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:54 smithi062 ceph-mon[90553]: failed to encode map e135 with expected crc 2024-03-24T11:27:54.445 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:54 smithi062 ceph-mon[90553]: failed to encode map e135 with expected crc 2024-03-24T11:27:54.445 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:54 smithi062 ceph-mon[90553]: failed to encode map e135 with expected crc 2024-03-24T11:27:54.445 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:54 smithi062 ceph-mon[90553]: failed to encode map e135 with expected crc 2024-03-24T11:27:54.445 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:54 smithi062 ceph-mon[98643]: pgmap v157: 161 pgs: 13 stale+active+clean, 148 active+clean; 585 KiB data, 173 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-24T11:27:54.445 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:54 smithi062 ceph-mon[98643]: failed to encode map e135 with expected crc 2024-03-24T11:27:54.445 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:54 smithi062 ceph-mon[98643]: failed to encode map e135 with expected crc 2024-03-24T11:27:54.445 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:54 smithi062 ceph-mon[98643]: osdmap e135: 8 total, 7 up, 8 in 2024-03-24T11:27:54.446 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:54 smithi062 ceph-mon[98643]: failed to encode map e135 with expected crc 2024-03-24T11:27:54.446 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:54 smithi062 ceph-mon[98643]: failed to encode map e135 with expected crc 2024-03-24T11:27:54.446 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:54 smithi062 ceph-mon[98643]: failed to encode map e135 with expected crc 2024-03-24T11:27:54.446 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:54 smithi062 ceph-mon[98643]: failed to encode map e135 with expected crc 2024-03-24T11:27:54.446 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:54 smithi062 ceph-mon[98643]: failed to encode map e135 with expected crc 2024-03-24T11:27:54.556 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:54 smithi139 ceph-mon[91894]: pgmap v157: 161 pgs: 13 stale+active+clean, 148 active+clean; 585 KiB data, 173 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-24T11:27:54.556 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:54 smithi139 ceph-mon[91894]: failed to encode map e135 with expected crc 2024-03-24T11:27:54.556 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:54 smithi139 ceph-mon[91894]: failed to encode map e135 with expected crc 2024-03-24T11:27:54.556 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:54 smithi139 ceph-mon[91894]: osdmap e135: 8 total, 7 up, 8 in 2024-03-24T11:27:54.556 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:54 smithi139 ceph-mon[91894]: failed to encode map e135 with expected crc 2024-03-24T11:27:54.556 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:54 smithi139 ceph-mon[91894]: failed to encode map e135 with expected crc 2024-03-24T11:27:54.556 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:54 smithi139 ceph-mon[91894]: failed to encode map e135 with expected crc 2024-03-24T11:27:54.556 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:54 smithi139 ceph-mon[91894]: failed to encode map e135 with expected crc 2024-03-24T11:27:54.556 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:54 smithi139 ceph-mon[91894]: failed to encode map e135 with expected crc 2024-03-24T11:27:54.977 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:54 smithi062 podman[137163]: 2024-03-24 11:27:54.719987228 +0000 UTC m=+0.622020898 container remove af198930cef743c48b960e06a1ba1de7d2b3e4f68efbea05633b6886f2545c52 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2-deactivate, RELEASE=squid-db0330b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T11:27:54.977 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:54 smithi062 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.2.service: Deactivated successfully. 2024-03-24T11:27:54.977 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:54 smithi062 systemd[1]: Stopped Ceph osd.2 for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:27:54.977 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:54 smithi062 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.2.service: Consumed 1.992s CPU time. 2024-03-24T11:27:54.977 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:54 smithi062 systemd[1]: Starting Ceph osd.2 for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:27:55.432 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:55 smithi062 ceph-mon[90553]: Health check failed: Degraded data redundancy: 5/630 objects degraded (0.794%), 1 pg degraded (PG_DEGRADED) 2024-03-24T11:27:55.433 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:55 smithi062 ceph-mon[98643]: Health check failed: Degraded data redundancy: 5/630 objects degraded (0.794%), 1 pg degraded (PG_DEGRADED) 2024-03-24T11:27:55.433 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:55 smithi062 podman[137281]: 2024-03-24 11:27:55.077840106 +0000 UTC m=+0.017782877 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:27:55.433 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:55 smithi062 podman[137281]: 2024-03-24 11:27:55.204450656 +0000 UTC m=+0.144393400 container create 394b3c2e8ecbed3ccfceb233933d92103dda510ae0138514bc02e9ce42ba142a (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2-activate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-db0330b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-24T11:27:55.433 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:55 smithi062 podman[137281]: 2024-03-24 11:27:55.432758925 +0000 UTC m=+0.372701692 container init 394b3c2e8ecbed3ccfceb233933d92103dda510ae0138514bc02e9ce42ba142a (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2-activate, RELEASE=squid-db0330b, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-24T11:27:55.577 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:55 smithi139 ceph-mon[91894]: Health check failed: Degraded data redundancy: 5/630 objects degraded (0.794%), 1 pg degraded (PG_DEGRADED) 2024-03-24T11:27:55.727 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:55 smithi062 podman[137281]: 2024-03-24 11:27:55.436337531 +0000 UTC m=+0.376280278 container start 394b3c2e8ecbed3ccfceb233933d92103dda510ae0138514bc02e9ce42ba142a (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2-activate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b) 2024-03-24T11:27:55.727 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:55 smithi062 podman[137281]: 2024-03-24 11:27:55.532829794 +0000 UTC m=+0.472772554 container attach 394b3c2e8ecbed3ccfceb233933d92103dda510ae0138514bc02e9ce42ba142a (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2-activate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=squid-db0330b) 2024-03-24T11:27:56.227 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:55 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2-activate[137296]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-24T11:27:56.228 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:55 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2-activate[137296]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-24T11:27:56.228 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:55 smithi062 bash[137281]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-24T11:27:56.228 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:55 smithi062 bash[137281]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-24T11:27:56.228 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:55 smithi062 bash[137281]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-24T11:27:56.228 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:55 smithi062 bash[137281]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-24T11:27:56.228 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:55 smithi062 bash[137281]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-03-24T11:27:56.228 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:55 smithi062 bash[137281]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-24T11:27:56.228 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:55 smithi062 bash[137281]: --> ceph-volume raw activate successful for osd ID: 2 2024-03-24T11:27:56.228 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:55 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2-activate[137296]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-24T11:27:56.228 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:55 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2-activate[137296]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-24T11:27:56.228 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:55 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2-activate[137296]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-03-24T11:27:56.228 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:55 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2-activate[137296]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-24T11:27:56.228 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:55 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2-activate[137296]: --> ceph-volume raw activate successful for osd ID: 2 2024-03-24T11:27:56.228 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:55 smithi062 podman[137281]: 2024-03-24 11:27:55.903427723 +0000 UTC m=+0.843370467 container died 394b3c2e8ecbed3ccfceb233933d92103dda510ae0138514bc02e9ce42ba142a (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-db0330b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-03-24T11:27:56.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:56 smithi139 ceph-mon[91894]: pgmap v159: 161 pgs: 1 active+undersized+degraded, 5 peering, 11 stale+active+clean, 6 active+undersized, 138 active+clean; 585 KiB data, 173 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 5/630 objects degraded (0.794%) 2024-03-24T11:27:56.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:56.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:27:56.629 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:56 smithi062 ceph-mon[90553]: pgmap v159: 161 pgs: 1 active+undersized+degraded, 5 peering, 11 stale+active+clean, 6 active+undersized, 138 active+clean; 585 KiB data, 173 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 5/630 objects degraded (0.794%) 2024-03-24T11:27:56.629 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:56.629 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:27:56.629 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:56 smithi062 ceph-mon[98643]: pgmap v159: 161 pgs: 1 active+undersized+degraded, 5 peering, 11 stale+active+clean, 6 active+undersized, 138 active+clean; 585 KiB data, 173 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 5/630 objects degraded (0.794%) 2024-03-24T11:27:56.629 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:56.629 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:27:56.967 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:56 smithi062 podman[137424]: 2024-03-24 11:27:56.628623317 +0000 UTC m=+0.720107416 container remove 394b3c2e8ecbed3ccfceb233933d92103dda510ae0138514bc02e9ce42ba142a (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2-activate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-db0330b, org.label-schema.build-date=20240307, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-24T11:27:56.968 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:56 smithi062 podman[137485]: 2024-03-24 11:27:56.741978066 +0000 UTC m=+0.017738653 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:27:56.968 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:56 smithi062 podman[137485]: 2024-03-24 11:27:56.863202401 +0000 UTC m=+0.138962969 container create 0eba40dc96b39a58e478f1b7059d1ef6041365fca7f5c1e4949e69c5807894dd (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-db0330b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=) 2024-03-24T11:27:57.410 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:57 smithi062 podman[137485]: 2024-03-24 11:27:57.12445962 +0000 UTC m=+0.400220191 container init 0eba40dc96b39a58e478f1b7059d1ef6041365fca7f5c1e4949e69c5807894dd (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-24T11:27:57.411 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:57 smithi062 podman[137485]: 2024-03-24 11:27:57.127658605 +0000 UTC m=+0.403419173 container start 0eba40dc96b39a58e478f1b7059d1ef6041365fca7f5c1e4949e69c5807894dd (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_CLEAN=True, RELEASE=squid-db0330b, io.buildah.version=1.33.5) 2024-03-24T11:27:57.411 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:57 smithi062 ceph-osd[137504]: -- 172.21.15.62:0/3800842781 <== mon.0 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ed321fb1e0 con 0x55ed321cd400 2024-03-24T11:27:57.411 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:57 smithi062 bash[137485]: 0eba40dc96b39a58e478f1b7059d1ef6041365fca7f5c1e4949e69c5807894dd 2024-03-24T11:27:57.411 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:57 smithi062 systemd[1]: Started Ceph osd.2 for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:27:57.977 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:27:57 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2[137500]: 2024-03-24T11:27:57.671+0000 7ff64477a7c0 -1 Falling back to public interface 2024-03-24T11:27:58.533 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[90553]: pgmap v160: 161 pgs: 12 active+undersized+degraded, 5 peering, 2 stale+active+clean, 27 active+undersized, 115 active+clean; 585 KiB data, 174 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 37/630 objects degraded (5.873%) 2024-03-24T11:27:58.534 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:58.534 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:58.534 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:58.534 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:58.534 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:58.534 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:58.534 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:58.534 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:58.535 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:58.535 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:58.535 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:58.535 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:58.535 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:58.535 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:58.535 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:58.535 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:58.535 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:58.535 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:58.535 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:58.535 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:58.535 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:58.535 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:58.535 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:58.536 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:58.536 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:58.536 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:58.536 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:58.536 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[98643]: pgmap v160: 161 pgs: 12 active+undersized+degraded, 5 peering, 2 stale+active+clean, 27 active+undersized, 115 active+clean; 585 KiB data, 174 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 37/630 objects degraded (5.873%) 2024-03-24T11:27:58.536 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:58.536 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:58.536 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:58.536 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:58.536 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:58.536 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:58.537 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:58.537 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:58.537 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:58.537 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:58.537 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:58.537 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:58.537 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:58.537 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:58.537 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:58.537 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:58.537 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:58.537 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:58.537 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:58.537 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:58.538 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:58.538 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:58.538 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:58.538 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:58.538 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:58.538 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:58.538 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:58.538 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:58.538 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:58.538 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:58.539 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:58.539 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:58.539 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:58.539 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:58.539 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:58.539 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:58.539 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:58.539 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:58.539 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:58.539 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:58.539 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:58.539 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:58.539 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:58.539 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:58.539 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:58.540 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:58.540 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:58.540 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:58.540 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:58.540 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:58.540 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:58.540 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:58.540 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:58.540 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:58.540 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:58.540 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:58.540 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:58.541 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:58.541 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:58.541 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:58.541 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:58.541 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:58.541 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:58.541 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:58.541 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:58.541 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:58.541 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:58.542 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:58.542 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:58.542 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:58.542 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:58.542 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:58.542 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:58.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:58 smithi139 ceph-mon[91894]: pgmap v160: 161 pgs: 12 active+undersized+degraded, 5 peering, 2 stale+active+clean, 27 active+undersized, 115 active+clean; 585 KiB data, 174 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 37/630 objects degraded (5.873%) 2024-03-24T11:27:58.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:58.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:58.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:58.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:58.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:58.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:58 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:58.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:58.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:58.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:58.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:58.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:58.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:58 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:58.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:58.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:58.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:58.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:58.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:58.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:58 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:58.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:58.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:58.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:58.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:58.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:58.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:58 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:58.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:58.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:58.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:58.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:58.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:58.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:58 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:58.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:58.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:58.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:58.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:58.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:58.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:58 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:58.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:58.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:58.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:58.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:58.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:58.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:58.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:58.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:58.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:58.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:58.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:58.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:58.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:58.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:59.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:59.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:59.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:59.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:59.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:59.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:59.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:59.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:59.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:59.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:59.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:59.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:59.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:59.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:59.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:59.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:59.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:59.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:59.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:59.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:59.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:59.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:59.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:59.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:59.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:59.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:59.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:59.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:59.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:59.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:59.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:59.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:59.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:59.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:59.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:59.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:59.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:59.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:59.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:59.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:59.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:59.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:59.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:59.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:59.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:59.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:59.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:59.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:59.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:59.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:59.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:59.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:59.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:59.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:59.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:59.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:59.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:59.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:59.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:59.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:59.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:59.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:59.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:59.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:59.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:59.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:59.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:59.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:59.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:59.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:59.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:59.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:59.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:59.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:59.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:59.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:59.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:59.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:59.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:59.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:59.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:59.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:59.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:59.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:59.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:59.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:59.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:59.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:59.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:59.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:59.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:59.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:59.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:59.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:59.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:59.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:59.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:59.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:59.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:59.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:59.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:59.489 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:59.490 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:59.490 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:59.490 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:59.490 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:59.490 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:59.490 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:59.490 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:59.490 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:59.490 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:59.491 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:59.491 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:59.491 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:59.491 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:59.491 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:59.491 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:59.491 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:59.491 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:59.491 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:59.491 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:59.491 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:27:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:59.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:59.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:59.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:59.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:59.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:59.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:59.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:59.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:59.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:59.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:59.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:59.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:59.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:59.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:59.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:59.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:59.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:59.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:59.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:59.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:59.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:59.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:59.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:59.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:59.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:59.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:59.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:59.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:59.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:59.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:59.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:59.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:59.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:59.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:59.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:59.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:59.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:59.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:59.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:59.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:59.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:59.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:59.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:59.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:59.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:59.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:27:59.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:59.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:59.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:59.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:59.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:59.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:59.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:59.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:59.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:59.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:27:59.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:27:59.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:27:59.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:27:59.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:27:59.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:27:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:00.428 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:00.428 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: pgmap v161: 161 pgs: 14 active+undersized+degraded, 32 active+undersized, 115 active+clean; 585 KiB data, 174 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 56/630 objects degraded (8.889%) 2024-03-24T11:28:00.428 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:00.428 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:00.428 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:00.428 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:00.428 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:00.428 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:00.428 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:00.428 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:00.428 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:00.428 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:00.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:00.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:00.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:00.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:00.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:00.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:00.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:00.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:00.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:00.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:00.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:00.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:00.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:00.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:00.429 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:00.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:00.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:00.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:00.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:00.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:00.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:00.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:00.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:00.431 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:00.431 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:00.431 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:00.431 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:00.431 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:00.431 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:00.431 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:00.431 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:00.431 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:00.431 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:00.431 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:00.431 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:00.431 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:00.431 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:00.431 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:00.432 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:00.432 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:00.432 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:00.432 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:00.432 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:00.432 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:00.432 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:00.432 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:00.432 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:00.433 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2[137500]: 2024-03-24T11:28:00.426+0000 7ff64477a7c0 -1 osd.2 133 log_to_monitors true 2024-03-24T11:28:00.433 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:00.433 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: pgmap v161: 161 pgs: 14 active+undersized+degraded, 32 active+undersized, 115 active+clean; 585 KiB data, 174 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 56/630 objects degraded (8.889%) 2024-03-24T11:28:00.433 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:00.433 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:00.433 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:00.433 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:00.434 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:00.434 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:00.434 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:00.434 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:00.434 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:00.434 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:00.434 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:00.434 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:00.434 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:00.434 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:00.434 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:00.434 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:00.434 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:00.434 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:00.435 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:00.435 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:00.435 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:00.435 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:00.435 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:00.435 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:00.435 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:00.435 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:00.435 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:00.435 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:00.435 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:00.435 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:00.435 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:00.435 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:00.435 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:00.436 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:00.436 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:00.436 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:00.436 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:00.436 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:00.436 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:00.436 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:00.436 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:00.436 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:00.436 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:00.436 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:00.436 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:00.436 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:00.436 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:00.436 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:00.437 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:00.437 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:00.437 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:00.437 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:00.437 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:00.437 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:00.437 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:00.437 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:00.437 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:00.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:00.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: pgmap v161: 161 pgs: 14 active+undersized+degraded, 32 active+undersized, 115 active+clean; 585 KiB data, 174 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 56/630 objects degraded (8.889%) 2024-03-24T11:28:00.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:00.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:00.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:00.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:00.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:00.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:00.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:00.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:00.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:00.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:00.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:00.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:00.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:00.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:00.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:00.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:00.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:00.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:00.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:00.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:00.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:00.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:00.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:00.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:00.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:00.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:00.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:00.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:00.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:00.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:00.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:00.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:00.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:00.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:00.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:00.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:00.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:00.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:00.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:00.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:00.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:00.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:00.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:00.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:00.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:00.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:00.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:00.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:00.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:00.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:00.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:00.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:00.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:00.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:00.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:00.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:00.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:00.795 INFO:teuthology.orchestra.run.smithi062.stdout:true 2024-03-24T11:28:01.066 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:28:00 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:28:00] "GET /metrics HTTP/1.1" 200 38515 "" "Prometheus/2.33.4" 2024-03-24T11:28:01.389 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:01.389 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:01.390 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:01.390 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:01.390 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:01.390 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:01.390 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:01.390 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:01.390 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: from='osd.2 [v2:172.21.15.62:6818/226191322,v1:172.21.15.62:6819/226191322]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-24T11:28:01.390 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:01.390 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:01.390 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:01.390 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:01.391 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:01.391 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:01.391 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:01.391 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:01.391 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:01.391 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:01.391 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:01.391 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:01.391 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:01.391 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:01.391 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:01.391 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:01.391 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:01.392 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:01.392 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:01.392 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:01.392 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:01.392 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:01.392 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:01.392 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:01.392 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: from='client.44388 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:28:01.392 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:01.392 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:01.393 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:01.393 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:01.393 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:01.393 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:01.393 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:01.393 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:01.393 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:01.393 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:01.393 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:01.393 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:01.393 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:01.393 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:01.394 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:01.394 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:01.394 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:01.394 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:01.394 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:01.394 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:01.394 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:01.395 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:01.395 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:01.395 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:01.395 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:01.395 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:01.395 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:01.395 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:01.395 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:01.395 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: from='osd.2 [v2:172.21.15.62:6818/226191322,v1:172.21.15.62:6819/226191322]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-24T11:28:01.395 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:01.395 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:01.395 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:01.396 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:01.396 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:01.396 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:01.396 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:01.396 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:01.396 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:01.396 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:01.396 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:01.396 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:01.396 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:01.396 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:01.396 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:01.397 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:01.397 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:01.397 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:01.397 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:01.397 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:01.397 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:01.397 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:01.397 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:01.397 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:01.398 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: from='client.44388 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:28:01.398 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:01.398 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:01.398 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:01.398 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:01.398 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:01.398 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:01.398 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:01.398 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:01.398 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:01.398 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:01.398 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:01.399 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:01.399 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:01.399 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:01.399 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:01.399 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:01.399 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:01.399 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:01.399 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:01.399 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:01.399 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:01.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:01.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:01.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:01.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:01.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:01.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:01.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:01.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:01.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: from='osd.2 [v2:172.21.15.62:6818/226191322,v1:172.21.15.62:6819/226191322]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-24T11:28:01.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:01.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:01.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:01.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:01.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:01.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:01.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:01.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:01.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:01.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:01.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:01.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:01.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:01.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:01.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:01.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:01.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:01.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:01.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:01.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:01.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:01.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:01.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:01.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:01.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: from='client.44388 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:28:01.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:01.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:01.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:01.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:01.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:01.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:01.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:01.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:01.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:01.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:01.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:01.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:01.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:01.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:01.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:01.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:01.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:01.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:01.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:01.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:01.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:01.637 INFO:teuthology.orchestra.run.smithi062.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T11:28:01.637 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi062 smithi062 running 12s ago 17m - - 2024-03-24T11:28:01.637 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi139 smithi139 running 1s ago 17m - - 2024-03-24T11:28:01.638 INFO:teuthology.orchestra.run.smithi062.stdout:alertmanager.a smithi062 *:9093,9094 running (7m) 12s ago 12m 24.6M - 0.23.0 ba2b418f427c 818298d58140 2024-03-24T11:28:01.638 INFO:teuthology.orchestra.run.smithi062.stdout:grafana.a smithi139 *:3000 running (6m) 1s ago 11m 53.9M - 8.3.5 dad864ee21e9 b48167e7171e 2024-03-24T11:28:01.638 INFO:teuthology.orchestra.run.smithi062.stdout:iscsi.foo.smithi062.dggsvk smithi062 running (5m) 12s ago 10m 60.3M - 3.5 e1d6a67b021e 0a98806ec839 2024-03-24T11:28:01.638 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.x smithi139 *:8443,9283,8765 running (4m) 1s ago 16m 451M - 19.0.0-1667-gdb0330b1 4f325f8440e9 d94987b11387 2024-03-24T11:28:01.638 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.y smithi062 *:8443,9283,8765 running (6m) 12s ago 18m 526M - 19.0.0-1667-gdb0330b1 4f325f8440e9 ca1a49028d65 2024-03-24T11:28:01.638 INFO:teuthology.orchestra.run.smithi062.stdout:mon.a smithi062 running (4m) 12s ago 18m 62.7M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 f7afe6e28c88 2024-03-24T11:28:01.638 INFO:teuthology.orchestra.run.smithi062.stdout:mon.b smithi139 running (3m) 1s ago 16m 41.8M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 ca357beea4dd 2024-03-24T11:28:01.638 INFO:teuthology.orchestra.run.smithi062.stdout:mon.c smithi062 running (3m) 12s ago 17m 52.7M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 459e7449ec39 2024-03-24T11:28:01.638 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.a smithi062 *:9100 running (7m) 12s ago 12m 17.3M - 1.3.1 1dbe0e931976 79d5a02a999f 2024-03-24T11:28:01.638 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.b smithi139 *:9100 running (6m) 1s ago 12m 15.6M - 1.3.1 1dbe0e931976 8f9f4608e465 2024-03-24T11:28:01.638 INFO:teuthology.orchestra.run.smithi062.stdout:osd.0 smithi062 running (117s) 12s ago 16m 53.4M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 dfe4920b49c4 2024-03-24T11:28:01.638 INFO:teuthology.orchestra.run.smithi062.stdout:osd.1 smithi062 running (68s) 12s ago 15m 44.5M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 c443b3fe97c1 2024-03-24T11:28:01.638 INFO:teuthology.orchestra.run.smithi062.stdout:osd.2 smithi062 starting - - - 4096M 2024-03-24T11:28:01.639 INFO:teuthology.orchestra.run.smithi062.stdout:osd.3 smithi062 running (15m) 12s ago 15m 62.6M 4096M 17.2.0 e1d6a67b021e 1d2868bd5195 2024-03-24T11:28:01.639 INFO:teuthology.orchestra.run.smithi062.stdout:osd.4 smithi139 running (14m) 1s ago 14m 66.8M 4096M 17.2.0 e1d6a67b021e 7568bfa9e74d 2024-03-24T11:28:01.639 INFO:teuthology.orchestra.run.smithi062.stdout:osd.5 smithi139 running (14m) 1s ago 14m 59.9M 4096M 17.2.0 e1d6a67b021e dd0abd599a14 2024-03-24T11:28:01.639 INFO:teuthology.orchestra.run.smithi062.stdout:osd.6 smithi139 running (14m) 1s ago 14m 58.5M 4096M 17.2.0 e1d6a67b021e a4b1289aca69 2024-03-24T11:28:01.639 INFO:teuthology.orchestra.run.smithi062.stdout:osd.7 smithi139 running (13m) 1s ago 13m 62.2M 4096M 17.2.0 e1d6a67b021e bc7fadbc1384 2024-03-24T11:28:01.639 INFO:teuthology.orchestra.run.smithi062.stdout:prometheus.a smithi139 *:9095 running (5m) 1s ago 12m 71.2M - 2.33.4 514e6a882f6e 2ed47f810d33 2024-03-24T11:28:01.639 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi062.sacpds smithi062 *:8000 running (11m) 12s ago 11m 74.1M - 17.2.0 e1d6a67b021e 89f4ac308e98 2024-03-24T11:28:01.639 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi139.yljesh smithi139 *:8000 running (11m) 1s ago 11m 73.3M - 17.2.0 e1d6a67b021e e189bdc12cae 2024-03-24T11:28:01.639 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi062.xvpxnk smithi062 *:80 running (11m) 12s ago 10m 74.0M - 17.2.0 e1d6a67b021e d1aa74adab44 2024-03-24T11:28:01.639 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi139.pbtrmp smithi139 *:80 running (10m) 1s ago 10m 76.9M - 17.2.0 e1d6a67b021e 7f8e336755d7 2024-03-24T11:28:01.727 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:28:01 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2[137500]: 2024-03-24T11:28:01.408+0000 7ff631bca700 -1 osd.2 133 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T11:28:02.119 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:28:02.119 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": { 2024-03-24T11:28:02.119 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 3 2024-03-24T11:28:02.119 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:28:02.120 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": { 2024-03-24T11:28:02.120 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-24T11:28:02.120 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:28:02.120 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": { 2024-03-24T11:28:02.120 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 5, 2024-03-24T11:28:02.120 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-24T11:28:02.120 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:28:02.120 INFO:teuthology.orchestra.run.smithi062.stdout: "rgw": { 2024-03-24T11:28:02.120 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4 2024-03-24T11:28:02.120 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:28:02.120 INFO:teuthology.orchestra.run.smithi062.stdout: "overall": { 2024-03-24T11:28:02.120 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 9, 2024-03-24T11:28:02.120 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 7 2024-03-24T11:28:02.120 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-24T11:28:02.121 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:28:02.513 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:28:02.514 INFO:teuthology.orchestra.run.smithi062.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69", 2024-03-24T11:28:02.514 INFO:teuthology.orchestra.run.smithi062.stdout: "in_progress": true, 2024-03-24T11:28:02.514 INFO:teuthology.orchestra.run.smithi062.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T11:28:02.514 INFO:teuthology.orchestra.run.smithi062.stdout: "services_complete": [ 2024-03-24T11:28:02.514 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr", 2024-03-24T11:28:02.514 INFO:teuthology.orchestra.run.smithi062.stdout: "mon" 2024-03-24T11:28:02.515 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-03-24T11:28:02.515 INFO:teuthology.orchestra.run.smithi062.stdout: "progress": "7/23 daemons upgraded", 2024-03-24T11:28:02.515 INFO:teuthology.orchestra.run.smithi062.stdout: "message": "Currently upgrading osd daemons", 2024-03-24T11:28:02.515 INFO:teuthology.orchestra.run.smithi062.stdout: "is_paused": false 2024-03-24T11:28:02.515 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:28:02.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:02 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:02.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:02 smithi139 ceph-mon[91894]: pgmap v162: 161 pgs: 14 active+undersized+degraded, 32 active+undersized, 115 active+clean; 585 KiB data, 174 MiB used, 715 GiB / 715 GiB avail; 577 B/s rd, 0 op/s; 56/630 objects degraded (8.889%) 2024-03-24T11:28:02.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:02 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:02.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:02 smithi139 ceph-mon[91894]: from='client.64360 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:28:02.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:02 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:02.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:02.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:02.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:02.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:02.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:02 smithi139 ceph-mon[91894]: from='osd.2 [v2:172.21.15.62:6818/226191322,v1:172.21.15.62:6819/226191322]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-24T11:28:02.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:02 smithi139 ceph-mon[91894]: failed to encode map e136 with expected crc 2024-03-24T11:28:02.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:02 smithi139 ceph-mon[91894]: osdmap e136: 8 total, 7 up, 8 in 2024-03-24T11:28:02.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:02 smithi139 ceph-mon[91894]: from='osd.2 [v2:172.21.15.62:6818/226191322,v1:172.21.15.62:6819/226191322]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:28:02.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:02.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:02 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:02.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:02.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:02.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:02.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:02.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:02.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:02 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:02.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:02 smithi139 ceph-mon[91894]: from='client.44400 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:28:02.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:02.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:02.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:02.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:02.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:02.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:02 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:02.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:02 smithi139 ceph-mon[91894]: failed to encode map e136 with expected crc 2024-03-24T11:28:02.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:02.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:02.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:02.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:02.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:02.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:02 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:02.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:02.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:02.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:02.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:02.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:02.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:02 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:02.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:02.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:02.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:02 smithi139 ceph-mon[91894]: from='client.? 172.21.15.62:0/1169626221' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:02.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:02.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:02.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:02.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:02.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:02.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:02.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:02.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:02.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[90553]: pgmap v162: 161 pgs: 14 active+undersized+degraded, 32 active+undersized, 115 active+clean; 585 KiB data, 174 MiB used, 715 GiB / 715 GiB avail; 577 B/s rd, 0 op/s; 56/630 objects degraded (8.889%) 2024-03-24T11:28:02.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:02.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[90553]: from='client.64360 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:28:02.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:02.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:02.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:02.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:02.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:02.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[90553]: from='osd.2 [v2:172.21.15.62:6818/226191322,v1:172.21.15.62:6819/226191322]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-24T11:28:02.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[90553]: failed to encode map e136 with expected crc 2024-03-24T11:28:02.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[90553]: osdmap e136: 8 total, 7 up, 8 in 2024-03-24T11:28:02.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[90553]: from='osd.2 [v2:172.21.15.62:6818/226191322,v1:172.21.15.62:6819/226191322]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:28:02.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:02.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:02.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:02.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:02.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:02.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:02.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:02.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:02.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[90553]: from='client.44400 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:28:02.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:02.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:02.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:02.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:02.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:02.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:02.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:02.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[98643]: pgmap v162: 161 pgs: 14 active+undersized+degraded, 32 active+undersized, 115 active+clean; 585 KiB data, 174 MiB used, 715 GiB / 715 GiB avail; 577 B/s rd, 0 op/s; 56/630 objects degraded (8.889%) 2024-03-24T11:28:02.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:02.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[98643]: from='client.64360 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:28:02.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:02.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:02.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:02.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:02.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:02.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[98643]: from='osd.2 [v2:172.21.15.62:6818/226191322,v1:172.21.15.62:6819/226191322]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-24T11:28:02.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[98643]: failed to encode map e136 with expected crc 2024-03-24T11:28:02.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[98643]: osdmap e136: 8 total, 7 up, 8 in 2024-03-24T11:28:02.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[98643]: from='osd.2 [v2:172.21.15.62:6818/226191322,v1:172.21.15.62:6819/226191322]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:28:02.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:02.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:02.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:02.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:02.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:02.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:02.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:02.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:02.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[98643]: from='client.44400 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:28:02.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:02.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:02.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:02.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:02.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:02.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:02.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[98643]: failed to encode map e136 with expected crc 2024-03-24T11:28:02.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:02.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:02.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:02.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:02.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:02.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:02.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:02.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:02.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:02.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:02.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:02.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:02.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:02.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:02.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[98643]: from='client.? 172.21.15.62:0/1169626221' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:02.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:02.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:02.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:02.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:02.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:02.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:02.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:02.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[90553]: failed to encode map e136 with expected crc 2024-03-24T11:28:02.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:02.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:02.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:02.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:02.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:02.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:02.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:02.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:02.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:02.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:02.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:02.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:02.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:02.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:02.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[90553]: from='client.? 172.21.15.62:0/1169626221' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:02.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:02.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:02.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:02.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:02.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:02.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:02.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:02.976 INFO:teuthology.orchestra.run.smithi062.stdout:HEALTH_WARN Degraded data redundancy: 56/630 objects degraded (8.889%), 14 pgs degraded 2024-03-24T11:28:02.976 INFO:teuthology.orchestra.run.smithi062.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 56/630 objects degraded (8.889%), 14 pgs degraded 2024-03-24T11:28:02.976 INFO:teuthology.orchestra.run.smithi062.stdout: pg 2.3 is active+undersized+degraded, acting [5,7] 2024-03-24T11:28:02.976 INFO:teuthology.orchestra.run.smithi062.stdout: pg 2.e is active+undersized+degraded, acting [3,7] 2024-03-24T11:28:02.976 INFO:teuthology.orchestra.run.smithi062.stdout: pg 2.16 is active+undersized+degraded, acting [5,6] 2024-03-24T11:28:02.976 INFO:teuthology.orchestra.run.smithi062.stdout: pg 3.0 is active+undersized+degraded, acting [1,6] 2024-03-24T11:28:02.976 INFO:teuthology.orchestra.run.smithi062.stdout: pg 3.4 is active+undersized+degraded, acting [1,5] 2024-03-24T11:28:02.976 INFO:teuthology.orchestra.run.smithi062.stdout: pg 3.5 is active+undersized+degraded, acting [5,3] 2024-03-24T11:28:02.977 INFO:teuthology.orchestra.run.smithi062.stdout: pg 3.9 is active+undersized+degraded, acting [4,7] 2024-03-24T11:28:02.977 INFO:teuthology.orchestra.run.smithi062.stdout: pg 3.13 is active+undersized+degraded, acting [7,4] 2024-03-24T11:28:02.977 INFO:teuthology.orchestra.run.smithi062.stdout: pg 3.1a is active+undersized+degraded, acting [4,1] 2024-03-24T11:28:02.977 INFO:teuthology.orchestra.run.smithi062.stdout: pg 3.1e is active+undersized+degraded, acting [3,6] 2024-03-24T11:28:02.977 INFO:teuthology.orchestra.run.smithi062.stdout: pg 3.1f is active+undersized+degraded, acting [0,5] 2024-03-24T11:28:02.977 INFO:teuthology.orchestra.run.smithi062.stdout: pg 4.d is active+undersized+degraded, acting [4,1] 2024-03-24T11:28:02.977 INFO:teuthology.orchestra.run.smithi062.stdout: pg 5.6 is active+undersized+degraded, acting [5,7] 2024-03-24T11:28:02.977 INFO:teuthology.orchestra.run.smithi062.stdout: pg 5.1c is active+undersized+degraded, acting [4,3] 2024-03-24T11:28:03.439 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: failed to encode map e136 with expected crc 2024-03-24T11:28:03.439 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:03.440 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:03.440 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:03.440 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:28:03.440 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:03.440 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:03.440 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:03.440 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:03.440 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T11:28:03.441 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:03.441 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:03.441 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: failed to encode map e137 with expected crc 2024-03-24T11:28:03.441 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: osd.2 [v2:172.21.15.62:6818/226191322,v1:172.21.15.62:6819/226191322] boot 2024-03-24T11:28:03.441 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: osdmap e137: 8 total, 8 up, 8 in 2024-03-24T11:28:03.441 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T11:28:03.441 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: failed to encode map e137 with expected crc 2024-03-24T11:28:03.441 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:03.441 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:03.442 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:03.442 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:03.442 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:03.442 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:03.442 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: from='client.64378 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:28:03.442 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:03.442 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:03.443 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:03.443 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:03.443 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:03.443 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:03.443 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:03.443 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:03.443 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:03.443 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:03.444 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:03.444 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:03.444 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:03.444 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:03.444 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:03.444 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:03.444 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:03.444 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:03.444 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: from='client.? 172.21.15.62:0/1764326865' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:28:03.444 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:03.444 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:03.445 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:03.445 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:03.445 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:03.445 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:03.445 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:03.445 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:03.445 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:03.445 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:03.445 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:03.445 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:03.445 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:03.445 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:03.445 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:03.446 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: failed to encode map e136 with expected crc 2024-03-24T11:28:03.446 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:03.446 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:03.446 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:03.446 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:28:03.446 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:03.446 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:03.446 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:03.446 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:03.447 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T11:28:03.447 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:03.447 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:03.447 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: failed to encode map e137 with expected crc 2024-03-24T11:28:03.447 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: osd.2 [v2:172.21.15.62:6818/226191322,v1:172.21.15.62:6819/226191322] boot 2024-03-24T11:28:03.447 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: osdmap e137: 8 total, 8 up, 8 in 2024-03-24T11:28:03.447 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T11:28:03.448 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: failed to encode map e137 with expected crc 2024-03-24T11:28:03.448 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:03.448 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:03.448 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:03.448 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:03.448 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:03.448 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:03.448 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: from='client.64378 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:28:03.449 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:03.449 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:03.449 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:03.449 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:03.449 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:03.449 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:03.450 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:03.450 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:03.450 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:03.450 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:03.450 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:03.450 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:03.451 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:03.451 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:03.451 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:03.451 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:03.451 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:03.451 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:03.451 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: from='client.? 172.21.15.62:0/1764326865' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:28:03.452 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:03.452 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:03.452 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:03.452 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:03.452 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:03.452 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:03.452 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:03.452 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:03.453 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:03.453 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:03.453 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:03.453 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:03.453 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:03.453 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:03.453 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:03.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: failed to encode map e136 with expected crc 2024-03-24T11:28:03.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:03.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:03.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:03.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:28:03.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:03.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:03.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:03.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:03.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T11:28:03.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:03.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:03.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: failed to encode map e137 with expected crc 2024-03-24T11:28:03.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: osd.2 [v2:172.21.15.62:6818/226191322,v1:172.21.15.62:6819/226191322] boot 2024-03-24T11:28:03.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: osdmap e137: 8 total, 8 up, 8 in 2024-03-24T11:28:03.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T11:28:03.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: failed to encode map e137 with expected crc 2024-03-24T11:28:03.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:03.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:03.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:03.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:03.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:03.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:03.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: from='client.64378 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:28:03.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:03.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:03.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:03.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:03.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:03.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:03.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:03.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:03.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:03.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:03.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:03.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:03.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:03.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:03.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:03.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:03.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:03.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:03.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: from='client.? 172.21.15.62:0/1764326865' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:28:03.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:03.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:03.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:03.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:03.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:03.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:03.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:03.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:03.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:03.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:03.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:03.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:03.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:03.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:03.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:04.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:04.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: pgmap v165: 161 pgs: 14 active+undersized+degraded, 32 active+undersized, 115 active+clean; 585 KiB data, 174 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 56/630 objects degraded (8.889%) 2024-03-24T11:28:04.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:04.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:04.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:04.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:04.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:04.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:04.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:04.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:04.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:04.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:04.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:04.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:04.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:04.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:04.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: failed to encode map e138 with expected crc 2024-03-24T11:28:04.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: osdmap e138: 8 total, 8 up, 8 in 2024-03-24T11:28:04.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: failed to encode map e138 with expected crc 2024-03-24T11:28:04.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: failed to encode map e138 with expected crc 2024-03-24T11:28:04.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: failed to encode map e138 with expected crc 2024-03-24T11:28:04.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: failed to encode map e138 with expected crc 2024-03-24T11:28:04.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: failed to encode map e138 with expected crc 2024-03-24T11:28:04.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: failed to encode map e138 with expected crc 2024-03-24T11:28:04.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: failed to encode map e138 with expected crc 2024-03-24T11:28:04.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:04.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:04.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:04.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:04.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:04.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:04.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:04.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:04.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:04.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:04.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:04.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:04.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:04.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:04.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:04.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:04.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:04.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:04.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:04.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:04.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:04.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:04.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:04.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:04.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:04.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:04.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:04.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:04.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:04.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:04.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:04.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:04.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:04.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:04.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:04.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:04.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:04.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:04.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:04.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:04.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:04.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:04.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:04.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:04.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:04.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:04.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:04.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:04.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:04.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:04.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:04.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:04 smithi139 ceph-mon[91894]: Health check update: Degraded data redundancy: 56/630 objects degraded (8.889%), 14 pgs degraded (PG_DEGRADED) 2024-03-24T11:28:04.614 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:04.614 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: pgmap v165: 161 pgs: 14 active+undersized+degraded, 32 active+undersized, 115 active+clean; 585 KiB data, 174 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 56/630 objects degraded (8.889%) 2024-03-24T11:28:04.614 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:04.614 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:04.615 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:04.615 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:04.615 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:04.615 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:04.615 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:04.615 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:04.615 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:04.615 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:04.615 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:04.615 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:04.615 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:04.615 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:04.615 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: failed to encode map e138 with expected crc 2024-03-24T11:28:04.615 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: osdmap e138: 8 total, 8 up, 8 in 2024-03-24T11:28:04.616 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: failed to encode map e138 with expected crc 2024-03-24T11:28:04.616 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: failed to encode map e138 with expected crc 2024-03-24T11:28:04.616 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: failed to encode map e138 with expected crc 2024-03-24T11:28:04.616 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: failed to encode map e138 with expected crc 2024-03-24T11:28:04.616 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: failed to encode map e138 with expected crc 2024-03-24T11:28:04.616 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: failed to encode map e138 with expected crc 2024-03-24T11:28:04.616 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: failed to encode map e138 with expected crc 2024-03-24T11:28:04.616 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:04.616 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:04.616 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:04.616 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:04.616 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:04.616 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:04.616 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:04.617 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:04.617 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: pgmap v165: 161 pgs: 14 active+undersized+degraded, 32 active+undersized, 115 active+clean; 585 KiB data, 174 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 56/630 objects degraded (8.889%) 2024-03-24T11:28:04.617 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:04.617 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:04.617 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:04.617 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:04.617 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:04.617 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:04.617 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:04.617 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:04.617 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:04.617 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:04.617 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:04.618 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:04.618 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:04.618 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:04.618 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: failed to encode map e138 with expected crc 2024-03-24T11:28:04.618 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: osdmap e138: 8 total, 8 up, 8 in 2024-03-24T11:28:04.618 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: failed to encode map e138 with expected crc 2024-03-24T11:28:04.618 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: failed to encode map e138 with expected crc 2024-03-24T11:28:04.618 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: failed to encode map e138 with expected crc 2024-03-24T11:28:04.618 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: failed to encode map e138 with expected crc 2024-03-24T11:28:04.618 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: failed to encode map e138 with expected crc 2024-03-24T11:28:04.618 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: failed to encode map e138 with expected crc 2024-03-24T11:28:04.618 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: failed to encode map e138 with expected crc 2024-03-24T11:28:04.618 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:04.618 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:04.618 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:04.619 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:04.619 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:04.619 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:04.619 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:04.619 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:04.619 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:04.619 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:04.619 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:04.620 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:04.620 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:04.620 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:04.620 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:04.620 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:04.620 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:04.620 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:04.620 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:04.620 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:04.620 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:04.620 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:04.620 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:04.620 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:04.620 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:04.620 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:04.621 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:04.621 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:04.621 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:04.621 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:04.621 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:04.621 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:04.621 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:04.621 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:04.621 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:04.621 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:04.621 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:04.621 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:04.621 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:04.621 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:04.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:04.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:04.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:04.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:04.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:04.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:04.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:04.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:04.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:04.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:04.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:04.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[90553]: Health check update: Degraded data redundancy: 56/630 objects degraded (8.889%), 14 pgs degraded (PG_DEGRADED) 2024-03-24T11:28:04.623 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:04.623 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:04.623 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:04.623 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:04.623 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:04.623 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:04.623 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:04.623 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:04.623 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:04.623 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:04.623 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:04.624 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:04.624 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:04.624 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:04.624 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:04.624 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:04.624 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:04.624 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:04.624 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:04.624 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:04.624 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:04.624 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:04.624 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:04.624 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:04.624 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:04.624 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:04.624 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:04.625 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:04.625 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:04.625 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:04.625 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:04.625 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:04.625 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:04.625 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:04.625 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:04.625 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:04.625 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:04.625 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:04.625 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:04.625 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:04.625 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:04.625 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:04.626 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:04.626 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:04.626 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:04 smithi062 ceph-mon[98643]: Health check update: Degraded data redundancy: 56/630 objects degraded (8.889%), 14 pgs degraded (PG_DEGRADED) 2024-03-24T11:28:05.520 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: failed to encode map e138 with expected crc 2024-03-24T11:28:05.520 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:05.520 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:05.520 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:05.520 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:05.520 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:05.520 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:05.520 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:05.520 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:05.521 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:05.521 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:05.521 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:05.521 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:05.521 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:05.521 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:05.521 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:05.521 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:05.521 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:05.521 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:05.521 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:05.521 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:05.521 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:05.521 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:05.521 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:05.522 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:05.522 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:05.522 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:05.522 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:05.522 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:05.522 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:05.522 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: failed to encode map e138 with expected crc 2024-03-24T11:28:05.522 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:05.522 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:05.522 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:05.523 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:05.523 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:05.523 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:05.523 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:05.523 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:05.523 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:05.523 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:05.523 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:05.523 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:05.523 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:05.523 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:05.523 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:05.523 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:05.523 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:05.523 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:05.524 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:05.524 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:05.524 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:05.524 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:05.524 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:05.524 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:05.524 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:05.524 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:05.524 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:05.524 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:05.524 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:05.525 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:05.525 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:05.525 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:05.525 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:05.525 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:05.525 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:05.525 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:05.525 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:05.525 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:05.525 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:05.525 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:05.525 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:05.525 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:05.526 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:05.526 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:05.526 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:05.526 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:05.526 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:05.526 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:05.526 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:05.526 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:05.526 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:05.526 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:05.526 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:05.526 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:05.526 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:05.527 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:05.527 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:05.527 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:05.527 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:05.527 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:05.527 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:05.527 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:05.527 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:05.527 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:05.527 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:05.527 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:05.528 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:05.528 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:05.528 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:05.528 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:05.528 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:05.528 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:05.528 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:05.528 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:05.528 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:05.528 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:05.529 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:05.529 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:05.529 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:05.529 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:05.529 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:05.529 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:05.529 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:05.529 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:05.529 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:05.529 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:05.529 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:05.530 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:05.530 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:05.530 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:05.530 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:05.530 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:05.530 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:05.530 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:05.530 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:05.530 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:05.530 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:05.530 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:05.530 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:05.530 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:05.531 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:05.531 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:05.531 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:05.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: failed to encode map e138 with expected crc 2024-03-24T11:28:05.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:05.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:05.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:05.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:05.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:05.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:05.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:05.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:05.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:05.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:05.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:05.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:05.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:05.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:05.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:05.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:05.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:05.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:05.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:05.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:05.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:05.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:05.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:05.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:05.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:05.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:05.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:05.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:05.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:05.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:05.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:05.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:05.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:05.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:05.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:05.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:05.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:05.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:05.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:05.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:05.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:05.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:05.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:05.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:05.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:05.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:05.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:05.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:05.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:05.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:05.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:05.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:05.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:05.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:05.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:05.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:05.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:05.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:05.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:05.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:05.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:05.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:05.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:05.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:05.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:05.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:06.473 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:06.473 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[90553]: pgmap v167: 161 pgs: 5 peering, 11 active+undersized+degraded, 25 active+undersized, 120 active+clean; 585 KiB data, 174 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 32/630 objects degraded (5.079%) 2024-03-24T11:28:06.473 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:06.473 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:06.473 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:06.473 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:06.473 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:06.473 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:06.473 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:06.473 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:06.473 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:06.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:06.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:06.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:06.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:06.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:06.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:06.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:06.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:06.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:06.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:06.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:06.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:06.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:06.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:06.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:06.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:06.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:06.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:06.475 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:06.475 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[98643]: pgmap v167: 161 pgs: 5 peering, 11 active+undersized+degraded, 25 active+undersized, 120 active+clean; 585 KiB data, 174 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 32/630 objects degraded (5.079%) 2024-03-24T11:28:06.475 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:06.475 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:06.475 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:06.475 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:06.476 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:06.476 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:06.476 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:06.476 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:06.476 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:06.476 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:06.476 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:06.476 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:06.476 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:06.476 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:06.476 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:06.476 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:06.476 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:06.476 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:06.476 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:06.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:06.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:06.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:06.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:06.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:06.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:06.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:06.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:06.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:06.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:06.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:06.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:06.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:06.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:06.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:06.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:06.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:06.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:06.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:06.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:06.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:06.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:06.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:06.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:06.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:06.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:06.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:06.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:06.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:06.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:06.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:06.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:06.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:06.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:06.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:06.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:06.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:06.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:06.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:06.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:06.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:06.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:06.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:06.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:06.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:06.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:06.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:06.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:06.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:06.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:06.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:06.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:06.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:06 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:06.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:06 smithi139 ceph-mon[91894]: pgmap v167: 161 pgs: 5 peering, 11 active+undersized+degraded, 25 active+undersized, 120 active+clean; 585 KiB data, 174 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 32/630 objects degraded (5.079%) 2024-03-24T11:28:06.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:06 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:06.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:06 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:06.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:06.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:06.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:06.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:06.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:06.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:06 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:06.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:06.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:06.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:06.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:06.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:06.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:06 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:06.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:06.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:06.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:06.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:06.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:06.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:06 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:06.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:06.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:06.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:06.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:06.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:06.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:06 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:06.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:06.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:06.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:06.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:06.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:06.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:06 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:06.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:06.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:06.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:06.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:06.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:06.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:06 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:06.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:06.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:06.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:06.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:06.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:06.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:06.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:06.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:06.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:06.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:06.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:07.353 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:07.353 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:07.353 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:07.353 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:07.353 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:07.353 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:07.353 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:07.353 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:07.353 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:07.353 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:07.353 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:07.353 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:07.354 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:07.354 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:07.354 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:07.354 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:07.354 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:07.354 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:07.354 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:07.354 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:07.354 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:07.354 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:07.354 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:07.354 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:07.355 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:07.355 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:07.355 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:07.355 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:07.355 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:07.355 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:07.355 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:07.355 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:07.356 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:07.356 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:07.356 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:07.356 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:07.356 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:07.356 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:07.356 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:07.356 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:07.356 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:07.356 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:07.356 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:07.356 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:07.356 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:07.356 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:07.357 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:07.357 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:07.357 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:07.357 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:07.357 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:07.357 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:07.357 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:07.357 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:07.357 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:07.357 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:07.357 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:07.357 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:07.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:07.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:07.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:07.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:07.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:07.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:07.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:07.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:07.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:07.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:07.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:07.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:07.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:07.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:07.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:07.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:07.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:07.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:07.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:07.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:07.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:07.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:07.360 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:07.360 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:07.360 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:07.360 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:07.360 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:07.360 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:07.360 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:07.360 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:07.360 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:07.360 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:07.360 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:07.361 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:07.361 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:07.361 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:07.361 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:07.361 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:07.361 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:07.361 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:07.361 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:07.361 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:07.362 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:07.362 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:07.362 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:07.362 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:07.362 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:07.362 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:07.362 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:07.362 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:07.362 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:07.362 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:07.362 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:07.362 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:07.362 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:07.363 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:07.363 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:07.363 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:07.363 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:07.363 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:07.363 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:07.363 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:07.363 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:07.363 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:07.363 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:07.363 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:07.363 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:07.363 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:07.363 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:07.363 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:07 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:07.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:07.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:07.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:07.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:07.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:07.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:07.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:07.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:07.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:07.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:07.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:07.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:07.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:07.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:07.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:07.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:07.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:07.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:07.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:07.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:07.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:07.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:07.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:07.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:07.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:07.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:07.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:07.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:07.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:07.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:07.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:07.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:07.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:07.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:07.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:07.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:07.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:07.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:07.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:07.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:07.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:07.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:07.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:07.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:07.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:07.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:07.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:07.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:07.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:07.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:07.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:07.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:07.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:07.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:07.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:07.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:07.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:07.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:07.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:07.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:07.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:07.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:07.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:07.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:07 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:08.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:08.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: pgmap v168: 161 pgs: 5 peering, 156 active+clean; 585 KiB data, 175 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-24T11:28:08.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:08.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:08.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:08.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:08.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:08.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:08.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:08.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:08.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:08.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:08.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:08.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:08.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:08.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:08.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:08.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:08.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:08.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:08.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:08.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:08.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:08.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:08.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:08.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:08.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:08.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:08.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:08.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:08.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:08.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: pgmap v168: 161 pgs: 5 peering, 156 active+clean; 585 KiB data, 175 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-24T11:28:08.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:08.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:08.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:08.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:08.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:08.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:08.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:08.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:08.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:08.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:08.481 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:08.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:08.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:08.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:08.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:08.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:08.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:08.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:08.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:08.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:08.482 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:08.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:08.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:08.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:08.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:08.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:08.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:08.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:08.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:08.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:08.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:08.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:08.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:08.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:08.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:08.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:08.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:08.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:08.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 32/630 objects degraded (5.079%), 11 pgs degraded) 2024-03-24T11:28:08.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: Cluster is now healthy 2024-03-24T11:28:08.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:08.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:08.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:08.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:08.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:08.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:08.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:08.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:08.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:08.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:08.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:08.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:08.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:08.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:08.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:08.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:08.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:08.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:08.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:08.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:08.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 32/630 objects degraded (5.079%), 11 pgs degraded) 2024-03-24T11:28:08.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: Cluster is now healthy 2024-03-24T11:28:08.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:08.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:08.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:08.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:08.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:08.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:08.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:08.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:08.487 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:08.488 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:08.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:08.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: pgmap v168: 161 pgs: 5 peering, 156 active+clean; 585 KiB data, 175 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-24T11:28:08.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:08.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:08.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:08.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:08.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:08.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:08.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:08.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:08.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:08.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:08.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:08.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:08.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:08.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:08.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:08.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:08.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:08.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:08.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:08.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:08.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:08.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:08.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:08.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:08.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:08.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:08.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:08.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:08.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:08.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:08.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:08.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:08.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:08.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:08.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:08.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:08.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:08.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:08.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 32/630 objects degraded (5.079%), 11 pgs degraded) 2024-03-24T11:28:08.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: Cluster is now healthy 2024-03-24T11:28:08.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:08.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:08.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:08.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:08.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:08.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:08.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:08.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:08.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:08.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:09.302 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:09.303 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:09.303 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:09.303 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:09.303 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:09.303 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:09.303 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:09.303 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:09.303 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:09.303 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:09.303 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:09.303 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:09.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:09.304 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:09.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:09.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:09.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:09.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:09.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:09.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:09.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:09.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:09.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:09.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:09.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:09.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:09.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:09.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:09.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:09.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:09.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:09.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:09.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:09.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:09.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:09.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:09.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:09.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:09.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:09.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:09.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:09.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:09.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:09.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:09.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:09.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:09.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:09.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:09.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:09.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:09.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:09.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:09.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:09.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:09.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:09.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:09.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:09.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:09.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:09.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:09.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:09.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:09.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:09.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:09.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:09.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:09.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:09.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:09.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:09.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:09.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:09.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:09.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:09.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:09.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:09.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:09.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:09.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:09 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-24T11:28:09.648 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:09.648 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:09.649 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:09.649 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:09.649 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:09.649 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:09.649 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:09.649 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:09.649 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:09.649 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:09.649 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:09.649 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:09.649 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:09.649 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:09.649 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:09.650 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:09.650 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:09.650 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:09.650 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:09.650 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:09.650 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:09.650 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:09.650 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:09.650 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:09.650 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:09.650 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:09.650 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:09.650 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:09.651 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:09.651 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:09.651 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:09.651 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:09.651 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:09.651 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:09.651 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:09.651 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:09.651 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:09.652 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:09.652 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:09.652 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:09.652 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:09.652 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:09.652 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:09.652 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:09.652 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:09.652 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:09.652 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:09.653 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:09.653 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:09.653 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:09.653 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:09.653 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:09.653 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:09.653 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:09.653 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:09.653 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:09.653 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:09.653 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:09.654 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:09.654 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:09.654 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:09.654 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:09.654 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:09.655 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:09.655 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:09.655 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:09.655 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:09.655 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:09.655 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:09.656 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:09.656 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:09.656 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:09.656 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:09.656 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:09.656 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:09.657 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:09.657 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:09.657 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:09.657 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:09.657 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:09.657 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-24T11:28:09.657 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:09.658 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:09.658 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:09.658 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:09.658 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:09.658 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:09.658 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:09.658 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:09.658 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:09.658 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:09.658 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:09.658 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:09.659 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:09.659 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:09.659 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:09.659 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:09.659 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:09.659 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:09.659 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:09.659 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:09.659 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:09.659 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:09.659 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:09.659 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:09.659 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:09.660 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:09.660 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:09.660 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:09.660 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:09.660 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:09.660 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:09.660 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:09.660 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:09 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-24T11:28:10.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:10 smithi062 ceph-mon[90553]: pgmap v169: 161 pgs: 161 active+clean; 585 KiB data, 175 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T11:28:10.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:10 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:10.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:10 smithi062 ceph-mon[90553]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-24T11:28:10.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:10 smithi062 ceph-mon[90553]: Upgrade: osd.3 is safe to restart 2024-03-24T11:28:10.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:10.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:10.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:10.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:10 smithi062 ceph-mon[98643]: pgmap v169: 161 pgs: 161 active+clean; 585 KiB data, 175 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T11:28:10.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:10 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:10.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:10 smithi062 ceph-mon[98643]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-24T11:28:10.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:10 smithi062 ceph-mon[98643]: Upgrade: osd.3 is safe to restart 2024-03-24T11:28:10.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:10.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:10.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:10.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:10 smithi139 ceph-mon[91894]: pgmap v169: 161 pgs: 161 active+clean; 585 KiB data, 175 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T11:28:10.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:10 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:10.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:10 smithi139 ceph-mon[91894]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-24T11:28:10.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:10 smithi139 ceph-mon[91894]: Upgrade: osd.3 is safe to restart 2024-03-24T11:28:10.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:10.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:10.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:11.096 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:28:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:28:10] "GET /metrics HTTP/1.1" 200 38515 "" "Prometheus/2.33.4" 2024-03-24T11:28:12.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:12 smithi062 ceph-mon[90553]: pgmap v170: 161 pgs: 161 active+clean; 585 KiB data, 175 MiB used, 715 GiB / 715 GiB avail; 587 B/s rd, 0 op/s 2024-03-24T11:28:12.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:12.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:28:12.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:12.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:12.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:12.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:12 smithi062 ceph-mon[90553]: Upgrade: Updating osd.3 2024-03-24T11:28:12.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:12.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-24T11:28:12.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:12.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:12 smithi062 ceph-mon[90553]: Deploying daemon osd.3 on smithi062 2024-03-24T11:28:12.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:12 smithi062 ceph-mon[98643]: pgmap v170: 161 pgs: 161 active+clean; 585 KiB data, 175 MiB used, 715 GiB / 715 GiB avail; 587 B/s rd, 0 op/s 2024-03-24T11:28:12.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:12.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:28:12.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:12.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:12.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:12.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:12 smithi062 ceph-mon[98643]: Upgrade: Updating osd.3 2024-03-24T11:28:12.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:12.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-24T11:28:12.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:12.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:12 smithi062 ceph-mon[98643]: Deploying daemon osd.3 on smithi062 2024-03-24T11:28:12.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:12 smithi139 ceph-mon[91894]: pgmap v170: 161 pgs: 161 active+clean; 585 KiB data, 175 MiB used, 715 GiB / 715 GiB avail; 587 B/s rd, 0 op/s 2024-03-24T11:28:12.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:12.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:28:12.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:12.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:12.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:12.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:12 smithi139 ceph-mon[91894]: Upgrade: Updating osd.3 2024-03-24T11:28:12.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:12.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-24T11:28:12.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:12.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:12 smithi139 ceph-mon[91894]: Deploying daemon osd.3 on smithi062 2024-03-24T11:28:13.462 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:13 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:28:13.462 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:13 smithi062 ceph-mon[98643]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:28:13.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:13 smithi139 ceph-mon[91894]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:28:14.382 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:14 smithi062 ceph-mon[90553]: pgmap v171: 161 pgs: 161 active+clean; 585 KiB data, 175 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T11:28:14.382 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:14 smithi062 ceph-mon[98643]: pgmap v171: 161 pgs: 161 active+clean; 585 KiB data, 175 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T11:28:14.496 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:14 smithi139 ceph-mon[91894]: pgmap v171: 161 pgs: 161 active+clean; 585 KiB data, 175 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T11:28:16.297 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:16 smithi062 ceph-mon[90553]: pgmap v172: 161 pgs: 161 active+clean; 585 KiB data, 175 MiB used, 715 GiB / 715 GiB avail; 874 B/s rd, 0 op/s 2024-03-24T11:28:16.297 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:16 smithi062 ceph-mon[98643]: pgmap v172: 161 pgs: 161 active+clean; 585 KiB data, 175 MiB used, 715 GiB / 715 GiB avail; 874 B/s rd, 0 op/s 2024-03-24T11:28:16.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:16 smithi139 ceph-mon[91894]: pgmap v172: 161 pgs: 161 active+clean; 585 KiB data, 175 MiB used, 715 GiB / 715 GiB avail; 874 B/s rd, 0 op/s 2024-03-24T11:28:17.227 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:16 smithi062 systemd[1]: Stopping Ceph osd.3 for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:28:17.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:17.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:17 smithi139 ceph-mon[91894]: osd.3 marked itself down and dead 2024-03-24T11:28:17.713 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:17.713 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:17 smithi062 ceph-mon[90553]: osd.3 marked itself down and dead 2024-03-24T11:28:17.713 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:17.713 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:17 smithi062 ceph-mon[98643]: osd.3 marked itself down and dead 2024-03-24T11:28:17.714 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:17 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3[41767]: 2024-03-24T11:28:17.233+0000 7f98b2f5a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T11:28:17.714 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:17 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3[41767]: 2024-03-24T11:28:17.233+0000 7f98b2f5a700 -1 osd.3 138 *** Got signal Terminated *** 2024-03-24T11:28:17.714 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:17 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3[41767]: 2024-03-24T11:28:17.233+0000 7f98b2f5a700 -1 osd.3 138 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T11:28:17.977 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:17 smithi062 podman[143354]: 2024-03-24 11:28:17.7133179 +0000 UTC m=+0.598241040 container died 1d2868bd519546612f8f79aecac93ba1d27fc65dfa8e080b18d37f809a523c9c (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3, io.openshift.tags=base centos centos-stream, maintainer=Guillaume Abrioux , GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, io.k8s.display-name=CentOS Stream 8, io.openshift.expose-services=, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, vcs-type=git, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., architecture=x86_64, build-date=2022-05-03T08:36:31.336870, CEPH_POINT_RELEASE=-17.2.0, version=8, release=754, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, com.redhat.component=centos-stream-container, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, distribution-scope=public, RELEASE=HEAD, name=centos-stream, io.buildah.version=1.19.8, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, GIT_BRANCH=HEAD, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, vendor=Red Hat, Inc., ceph=True) 2024-03-24T11:28:18.260 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:18 smithi062 podman[143354]: 2024-03-24 11:28:18.050842577 +0000 UTC m=+0.935765735 container cleanup 1d2868bd519546612f8f79aecac93ba1d27fc65dfa8e080b18d37f809a523c9c (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.openshift.tags=base centos centos-stream, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, RELEASE=HEAD, vendor=Red Hat, Inc., io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., GIT_REPO=https://github.com/ceph/ceph-container.git, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, io.k8s.display-name=CentOS Stream 8, io.openshift.expose-services=, name=centos-stream, io.buildah.version=1.19.8, build-date=2022-05-03T08:36:31.336870, version=8, ceph=True, CEPH_POINT_RELEASE=-17.2.0, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, GIT_BRANCH=HEAD, release=754, com.redhat.component=centos-stream-container, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, distribution-scope=public, vcs-type=git, architecture=x86_64) 2024-03-24T11:28:18.260 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:18 smithi062 bash[143354]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3 2024-03-24T11:28:18.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:18 smithi139 ceph-mon[91894]: pgmap v173: 161 pgs: 161 active+clean; 585 KiB data, 175 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:28:18.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:18 smithi139 ceph-mon[91894]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T11:28:18.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:18 smithi139 ceph-mon[91894]: failed to encode map e139 with expected crc 2024-03-24T11:28:18.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:18 smithi139 ceph-mon[91894]: failed to encode map e139 with expected crc 2024-03-24T11:28:18.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:18 smithi139 ceph-mon[91894]: osdmap e139: 8 total, 7 up, 8 in 2024-03-24T11:28:18.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:18 smithi139 ceph-mon[91894]: failed to encode map e139 with expected crc 2024-03-24T11:28:18.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:18 smithi139 ceph-mon[91894]: failed to encode map e139 with expected crc 2024-03-24T11:28:18.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:18 smithi139 ceph-mon[91894]: failed to encode map e139 with expected crc 2024-03-24T11:28:18.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:18 smithi139 ceph-mon[91894]: failed to encode map e139 with expected crc 2024-03-24T11:28:18.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:18 smithi139 ceph-mon[91894]: failed to encode map e139 with expected crc 2024-03-24T11:28:18.670 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:18 smithi062 podman[143371]: 2024-03-24 11:28:18.260731363 +0000 UTC m=+0.548222308 container remove 1d2868bd519546612f8f79aecac93ba1d27fc65dfa8e080b18d37f809a523c9c (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3, architecture=x86_64, GIT_BRANCH=HEAD, distribution-scope=public, com.redhat.component=centos-stream-container, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, CEPH_POINT_RELEASE=-17.2.0, version=8, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, RELEASE=HEAD, io.openshift.tags=base centos centos-stream, build-date=2022-05-03T08:36:31.336870, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, ceph=True, io.buildah.version=1.19.8, GIT_REPO=https://github.com/ceph/ceph-container.git, release=754, io.k8s.display-name=CentOS Stream 8, maintainer=Guillaume Abrioux , url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, io.openshift.expose-services=, vendor=Red Hat, Inc., vcs-type=git, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, GIT_CLEAN=True, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, name=centos-stream) 2024-03-24T11:28:18.670 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:18 smithi062 ceph-mon[90553]: pgmap v173: 161 pgs: 161 active+clean; 585 KiB data, 175 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:28:18.670 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:18 smithi062 ceph-mon[90553]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T11:28:18.670 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:18 smithi062 ceph-mon[90553]: failed to encode map e139 with expected crc 2024-03-24T11:28:18.671 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:18 smithi062 ceph-mon[90553]: failed to encode map e139 with expected crc 2024-03-24T11:28:18.671 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:18 smithi062 ceph-mon[90553]: osdmap e139: 8 total, 7 up, 8 in 2024-03-24T11:28:18.671 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:18 smithi062 ceph-mon[90553]: failed to encode map e139 with expected crc 2024-03-24T11:28:18.671 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:18 smithi062 ceph-mon[90553]: failed to encode map e139 with expected crc 2024-03-24T11:28:18.671 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:18 smithi062 ceph-mon[90553]: failed to encode map e139 with expected crc 2024-03-24T11:28:18.671 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:18 smithi062 ceph-mon[90553]: failed to encode map e139 with expected crc 2024-03-24T11:28:18.671 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:18 smithi062 ceph-mon[90553]: failed to encode map e139 with expected crc 2024-03-24T11:28:18.671 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:18 smithi062 ceph-mon[98643]: pgmap v173: 161 pgs: 161 active+clean; 585 KiB data, 175 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:28:18.671 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:18 smithi062 ceph-mon[98643]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T11:28:18.671 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:18 smithi062 ceph-mon[98643]: failed to encode map e139 with expected crc 2024-03-24T11:28:18.671 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:18 smithi062 ceph-mon[98643]: failed to encode map e139 with expected crc 2024-03-24T11:28:18.672 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:18 smithi062 ceph-mon[98643]: osdmap e139: 8 total, 7 up, 8 in 2024-03-24T11:28:18.672 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:18 smithi062 ceph-mon[98643]: failed to encode map e139 with expected crc 2024-03-24T11:28:18.672 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:18 smithi062 ceph-mon[98643]: failed to encode map e139 with expected crc 2024-03-24T11:28:18.672 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:18 smithi062 ceph-mon[98643]: failed to encode map e139 with expected crc 2024-03-24T11:28:18.672 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:18 smithi062 ceph-mon[98643]: failed to encode map e139 with expected crc 2024-03-24T11:28:18.672 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:18 smithi062 ceph-mon[98643]: failed to encode map e139 with expected crc 2024-03-24T11:28:18.964 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:18 smithi062 podman[143462]: 2024-03-24 11:28:18.670246682 +0000 UTC m=+0.109606028 container create 53b1b13430caf3affb9c93796846048c1c3e34ad92ce36c1bbd6a88578d5b2a7 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-db0330b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-24T11:28:18.964 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:18 smithi062 podman[143462]: 2024-03-24 11:28:18.57863741 +0000 UTC m=+0.017996759 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:28:18.964 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:18 smithi062 podman[143462]: 2024-03-24 11:28:18.889147736 +0000 UTC m=+0.328507097 container init 53b1b13430caf3affb9c93796846048c1c3e34ad92ce36c1bbd6a88578d5b2a7 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-db0330b, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-24T11:28:18.964 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:18 smithi062 podman[143462]: 2024-03-24 11:28:18.892238468 +0000 UTC m=+0.331597824 container start 53b1b13430caf3affb9c93796846048c1c3e34ad92ce36c1bbd6a88578d5b2a7 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-db0330b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-24T11:28:18.965 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:18 smithi062 podman[143462]: 2024-03-24 11:28:18.964312902 +0000 UTC m=+0.403672265 container attach 53b1b13430caf3affb9c93796846048c1c3e34ad92ce36c1bbd6a88578d5b2a7 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_CLEAN=True) 2024-03-24T11:28:19.227 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:19 smithi062 podman[143462]: 2024-03-24 11:28:19.193886646 +0000 UTC m=+0.633246003 container died 53b1b13430caf3affb9c93796846048c1c3e34ad92ce36c1bbd6a88578d5b2a7 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, RELEASE=squid-db0330b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-24T11:28:19.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:19 smithi139 ceph-mon[91894]: failed to encode map e140 with expected crc 2024-03-24T11:28:19.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:19 smithi139 ceph-mon[91894]: failed to encode map e140 with expected crc 2024-03-24T11:28:19.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:19 smithi139 ceph-mon[91894]: osdmap e140: 8 total, 7 up, 8 in 2024-03-24T11:28:19.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:19 smithi139 ceph-mon[91894]: failed to encode map e140 with expected crc 2024-03-24T11:28:19.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:19 smithi139 ceph-mon[91894]: failed to encode map e140 with expected crc 2024-03-24T11:28:19.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:19 smithi139 ceph-mon[91894]: failed to encode map e140 with expected crc 2024-03-24T11:28:19.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:19 smithi139 ceph-mon[91894]: failed to encode map e140 with expected crc 2024-03-24T11:28:19.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:19 smithi139 ceph-mon[91894]: failed to encode map e140 with expected crc 2024-03-24T11:28:19.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:19 smithi062 ceph-mon[90553]: failed to encode map e140 with expected crc 2024-03-24T11:28:19.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:19 smithi062 ceph-mon[90553]: failed to encode map e140 with expected crc 2024-03-24T11:28:19.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:19 smithi062 ceph-mon[90553]: osdmap e140: 8 total, 7 up, 8 in 2024-03-24T11:28:19.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:19 smithi062 ceph-mon[90553]: failed to encode map e140 with expected crc 2024-03-24T11:28:19.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:19 smithi062 ceph-mon[90553]: failed to encode map e140 with expected crc 2024-03-24T11:28:19.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:19 smithi062 ceph-mon[90553]: failed to encode map e140 with expected crc 2024-03-24T11:28:19.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:19 smithi062 ceph-mon[90553]: failed to encode map e140 with expected crc 2024-03-24T11:28:19.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:19 smithi062 ceph-mon[90553]: failed to encode map e140 with expected crc 2024-03-24T11:28:19.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:19 smithi062 ceph-mon[98643]: failed to encode map e140 with expected crc 2024-03-24T11:28:19.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:19 smithi062 ceph-mon[98643]: failed to encode map e140 with expected crc 2024-03-24T11:28:19.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:19 smithi062 ceph-mon[98643]: osdmap e140: 8 total, 7 up, 8 in 2024-03-24T11:28:19.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:19 smithi062 ceph-mon[98643]: failed to encode map e140 with expected crc 2024-03-24T11:28:19.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:19 smithi062 ceph-mon[98643]: failed to encode map e140 with expected crc 2024-03-24T11:28:19.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:19 smithi062 ceph-mon[98643]: failed to encode map e140 with expected crc 2024-03-24T11:28:19.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:19 smithi062 ceph-mon[98643]: failed to encode map e140 with expected crc 2024-03-24T11:28:19.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:19 smithi062 ceph-mon[98643]: failed to encode map e140 with expected crc 2024-03-24T11:28:20.226 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:19 smithi062 podman[143489]: 2024-03-24 11:28:19.77630741 +0000 UTC m=+0.576430183 container remove 53b1b13430caf3affb9c93796846048c1c3e34ad92ce36c1bbd6a88578d5b2a7 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=squid-db0330b, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307) 2024-03-24T11:28:20.227 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:19 smithi062 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.3.service: Deactivated successfully. 2024-03-24T11:28:20.227 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:19 smithi062 systemd[1]: Stopped Ceph osd.3 for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:28:20.227 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:19 smithi062 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.3.service: Consumed 10.650s CPU time. 2024-03-24T11:28:20.227 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:20 smithi062 systemd[1]: Starting Ceph osd.3 for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:28:20.477 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:20 smithi062 podman[143593]: 2024-03-24 11:28:20.1292801 +0000 UTC m=+0.018457988 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:28:20.477 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:20 smithi062 podman[143593]: 2024-03-24 11:28:20.319952075 +0000 UTC m=+0.209129956 container create 74b3b7d4a6ad68b88a85a275aecc43447e9059c54d7144f4930193c4c6c14fea (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3-activate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=squid-db0330b, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS) 2024-03-24T11:28:20.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:20 smithi062 ceph-mon[98643]: pgmap v176: 161 pgs: 5 active+undersized+degraded, 13 peering, 15 stale+active+clean, 9 active+undersized, 119 active+clean; 585 KiB data, 175 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 0 op/s; 22/630 objects degraded (3.492%) 2024-03-24T11:28:20.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:20 smithi062 ceph-mon[98643]: Health check failed: Reduced data availability: 5 pgs inactive, 6 pgs peering (PG_AVAILABILITY) 2024-03-24T11:28:20.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:20 smithi062 ceph-mon[98643]: Health check failed: Degraded data redundancy: 22/630 objects degraded (3.492%), 5 pgs degraded (PG_DEGRADED) 2024-03-24T11:28:20.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:20.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:20.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:20.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:20 smithi062 ceph-mon[90553]: pgmap v176: 161 pgs: 5 active+undersized+degraded, 13 peering, 15 stale+active+clean, 9 active+undersized, 119 active+clean; 585 KiB data, 175 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 0 op/s; 22/630 objects degraded (3.492%) 2024-03-24T11:28:20.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:20 smithi062 ceph-mon[90553]: Health check failed: Reduced data availability: 5 pgs inactive, 6 pgs peering (PG_AVAILABILITY) 2024-03-24T11:28:20.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:20 smithi062 ceph-mon[90553]: Health check failed: Degraded data redundancy: 22/630 objects degraded (3.492%), 5 pgs degraded (PG_DEGRADED) 2024-03-24T11:28:20.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:20.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:20.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:20.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:20 smithi139 ceph-mon[91894]: pgmap v176: 161 pgs: 5 active+undersized+degraded, 13 peering, 15 stale+active+clean, 9 active+undersized, 119 active+clean; 585 KiB data, 175 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 0 op/s; 22/630 objects degraded (3.492%) 2024-03-24T11:28:20.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:20 smithi139 ceph-mon[91894]: Health check failed: Reduced data availability: 5 pgs inactive, 6 pgs peering (PG_AVAILABILITY) 2024-03-24T11:28:20.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:20 smithi139 ceph-mon[91894]: Health check failed: Degraded data redundancy: 22/630 objects degraded (3.492%), 5 pgs degraded (PG_DEGRADED) 2024-03-24T11:28:20.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:20.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:20.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:20.805 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:20 smithi062 podman[143593]: 2024-03-24 11:28:20.530566709 +0000 UTC m=+0.419744589 container init 74b3b7d4a6ad68b88a85a275aecc43447e9059c54d7144f4930193c4c6c14fea (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3-activate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=squid-db0330b, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T11:28:20.806 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:20 smithi062 podman[143593]: 2024-03-24 11:28:20.533599178 +0000 UTC m=+0.422777061 container start 74b3b7d4a6ad68b88a85a275aecc43447e9059c54d7144f4930193c4c6c14fea (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3-activate, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2) 2024-03-24T11:28:20.806 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:20 smithi062 podman[143593]: 2024-03-24 11:28:20.605637658 +0000 UTC m=+0.494815539 container attach 74b3b7d4a6ad68b88a85a275aecc43447e9059c54d7144f4930193c4c6c14fea (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3-activate, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-db0330b) 2024-03-24T11:28:21.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:28:20 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:28:20] "GET /metrics HTTP/1.1" 200 38506 "" "Prometheus/2.33.4" 2024-03-24T11:28:21.227 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:20 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3-activate[143608]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-24T11:28:21.227 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:20 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3-activate[143608]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_1 2024-03-24T11:28:21.227 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:20 smithi062 bash[143593]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-24T11:28:21.227 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:20 smithi062 bash[143593]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_1 2024-03-24T11:28:21.227 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:20 smithi062 bash[143593]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_1 2024-03-24T11:28:21.227 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:20 smithi062 bash[143593]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-0 2024-03-24T11:28:21.227 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:20 smithi062 bash[143593]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-3/block 2024-03-24T11:28:21.227 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:20 smithi062 bash[143593]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-24T11:28:21.228 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:20 smithi062 bash[143593]: --> ceph-volume raw activate successful for osd ID: 3 2024-03-24T11:28:21.228 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:20 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3-activate[143608]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_1 2024-03-24T11:28:21.228 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:20 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3-activate[143608]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-0 2024-03-24T11:28:21.228 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:20 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3-activate[143608]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-3/block 2024-03-24T11:28:21.228 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:20 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3-activate[143608]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-24T11:28:21.228 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:20 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3-activate[143608]: --> ceph-volume raw activate successful for osd ID: 3 2024-03-24T11:28:21.228 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:21 smithi062 podman[143593]: 2024-03-24 11:28:21.00411609 +0000 UTC m=+0.893293973 container died 74b3b7d4a6ad68b88a85a275aecc43447e9059c54d7144f4930193c4c6c14fea (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3-activate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-db0330b, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T11:28:21.900 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:21 smithi062 podman[143741]: 2024-03-24 11:28:21.593020189 +0000 UTC m=+0.584052954 container remove 74b3b7d4a6ad68b88a85a275aecc43447e9059c54d7144f4930193c4c6c14fea (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3-activate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240307, RELEASE=squid-db0330b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-24T11:28:21.900 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:21 smithi062 podman[143805]: 2024-03-24 11:28:21.794082393 +0000 UTC m=+0.103622964 container create 4c055d2ab5c11100ccf0829ef77e12e9095c9a5038dff3584f90d694587a6c45 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3, RELEASE=squid-db0330b, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-24T11:28:21.900 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:21 smithi062 podman[143805]: 2024-03-24 11:28:21.708205043 +0000 UTC m=+0.017745633 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:28:22.285 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:21 smithi062 podman[143805]: 2024-03-24 11:28:21.980654287 +0000 UTC m=+0.290194870 container init 4c055d2ab5c11100ccf0829ef77e12e9095c9a5038dff3584f90d694587a6c45 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True) 2024-03-24T11:28:22.285 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:21 smithi062 podman[143805]: 2024-03-24 11:28:21.983646113 +0000 UTC m=+0.293186692 container start 4c055d2ab5c11100ccf0829ef77e12e9095c9a5038dff3584f90d694587a6c45 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-db0330b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-24T11:28:22.285 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:22 smithi062 ceph-osd[143824]: -- 172.21.15.62:0/2966145965 <== mon.1 v2:172.21.15.62:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ccbe5951e0 con 0x55ccbe567400 2024-03-24T11:28:22.285 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:22 smithi062 bash[143805]: 4c055d2ab5c11100ccf0829ef77e12e9095c9a5038dff3584f90d694587a6c45 2024-03-24T11:28:22.285 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:22 smithi062 systemd[1]: Started Ceph osd.3 for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:28:22.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:22 smithi139 ceph-mon[91894]: pgmap v177: 161 pgs: 5 active+undersized+degraded, 13 peering, 15 stale+active+clean, 9 active+undersized, 119 active+clean; 585 KiB data, 175 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s; 22/630 objects degraded (3.492%) 2024-03-24T11:28:22.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:22.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:22.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:22.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:22.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:22.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:22.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:22.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:22.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:22.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:22.727 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:22 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3[143820]: 2024-03-24T11:28:22.517+0000 7fb609cf47c0 -1 Falling back to public interface 2024-03-24T11:28:22.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:22 smithi062 ceph-mon[90553]: pgmap v177: 161 pgs: 5 active+undersized+degraded, 13 peering, 15 stale+active+clean, 9 active+undersized, 119 active+clean; 585 KiB data, 175 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s; 22/630 objects degraded (3.492%) 2024-03-24T11:28:22.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:22.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:22.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:22.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:22.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:22.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:22.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:22.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:22.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:22.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:22.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:22 smithi062 ceph-mon[98643]: pgmap v177: 161 pgs: 5 active+undersized+degraded, 13 peering, 15 stale+active+clean, 9 active+undersized, 119 active+clean; 585 KiB data, 175 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s; 22/630 objects degraded (3.492%) 2024-03-24T11:28:22.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:22.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:22.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:22.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:22.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:22.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:22.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:22.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:22.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:22.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:23.469 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:23.470 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:23.470 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[98643]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:28:23.470 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:23.470 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:23.470 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:23.470 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:23.470 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:23.470 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:23.470 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:23.470 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:23.470 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:23.471 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:23.471 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:23.471 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:23.471 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:23.471 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:23.471 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:23.471 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:23.471 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:23.471 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:23.471 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:23.471 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:23.471 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:23.471 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:23.471 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:23.471 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:23.472 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:23.472 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:23.472 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:23.472 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:23.472 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:23.472 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:23.472 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:23.472 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:23.472 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:23.472 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:23.472 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:23.472 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:23.472 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:23.472 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:23.473 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:23.473 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:23.473 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:23.473 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:23.473 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:23.473 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:23.473 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:23.473 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:23.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:23.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:23.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:28:23.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:23.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:23.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:23.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:23.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:23.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:23.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:23.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:23.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:23.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:23.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:23.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:23.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:23.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:23.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:23.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:23.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:23.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:23.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:23.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:23.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:23.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:23.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:23.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:23.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:23.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:23.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:23.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:23.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:23.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:23.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:23.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:23.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:23.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:23.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:23.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:23.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:23.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:23.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:23.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:23.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:23.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:23.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:23.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:23.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:23.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:23.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:23 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:23.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:23 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:23.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:23 smithi139 ceph-mon[91894]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:28:23.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:23.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:23.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:23.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:23.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:23.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:23 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:23.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:23.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:23.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:23.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:23.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:23.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:23 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:23.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:23.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:23.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:23.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:23.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:23.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:23 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:23.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:23.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:23.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:23.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:23.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:23.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:23 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:23.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:23.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:23.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:23.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:23.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:23.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:23 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:23.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:23.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:23.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:23.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:23.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:23.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:23 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:23.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:23.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:23.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:23.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:23.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:23.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:23.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:23.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:23.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:23.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:24.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: pgmap v178: 161 pgs: 18 active+undersized+degraded, 13 peering, 35 active+undersized, 95 active+clean; 585 KiB data, 176 MiB used, 715 GiB / 715 GiB avail; 60/630 objects degraded (9.524%) 2024-03-24T11:28:24.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:24.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:24.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:24.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:24.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:24.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:24.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:24.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:24.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:24.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:24.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:24.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:24.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:24.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:24.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:24.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:24.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:24.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:24.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:24.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:24.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:24.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:24.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:24.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:24.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:24.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:24.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:24.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:24.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:24.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:24.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:24.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:24.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:24.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:24.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:24.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:24.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:24.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:24.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:24.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:24.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:24.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:24.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:24.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:24.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:24.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:24.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:24.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:24.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:24.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:24.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:24.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:24.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:24.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:24.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:24.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:24.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:24.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:24.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:24.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:24.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:24.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:24.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:24.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:24.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:24.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:24 smithi139 ceph-mon[91894]: Health check update: Degraded data redundancy: 60/630 objects degraded (9.524%), 18 pgs degraded (PG_DEGRADED) 2024-03-24T11:28:24.614 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: pgmap v178: 161 pgs: 18 active+undersized+degraded, 13 peering, 35 active+undersized, 95 active+clean; 585 KiB data, 176 MiB used, 715 GiB / 715 GiB avail; 60/630 objects degraded (9.524%) 2024-03-24T11:28:24.614 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:24.614 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:24.614 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:24.614 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:24.614 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:24.614 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:24.614 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:24.614 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:24.614 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:24.614 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:24.614 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:24.614 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:24.614 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:24.615 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:24.615 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:24.615 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:24.615 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:24.615 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:24.615 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:24.615 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:24.615 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:24.615 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:24.615 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:24.615 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:24.615 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:24.615 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:24.615 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:24.615 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:24.616 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: pgmap v178: 161 pgs: 18 active+undersized+degraded, 13 peering, 35 active+undersized, 95 active+clean; 585 KiB data, 176 MiB used, 715 GiB / 715 GiB avail; 60/630 objects degraded (9.524%) 2024-03-24T11:28:24.616 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:24.616 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:24.616 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:24.616 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:24.616 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:24.616 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:24.617 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:24.617 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:24.617 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:24.617 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:24.617 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:24.617 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:24.617 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:24.617 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:24.617 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:24.617 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:24.617 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:24.617 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:24.617 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:24.617 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:24.617 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:24.618 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:24.618 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:24.618 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:24.618 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:24.618 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:24.618 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:24.618 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:24.618 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:24.618 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:24.618 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:24.618 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:24.618 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:24.618 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:24.618 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:24.618 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:24.619 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:24.619 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:24.619 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:24.619 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:24.619 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:24.619 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:24.619 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:24.619 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:24.619 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:24.619 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:24.619 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:24.619 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:24.619 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:24.619 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:24.619 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:24.620 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:24.620 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:24.620 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:24.620 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:24.620 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:24.620 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:24.620 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:24.620 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:24.620 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:24.620 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:24.620 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:24.620 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:24.620 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:24.621 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:24.621 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:24.621 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:24.621 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:24.621 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:24.621 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:24.621 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:24.621 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:24.621 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:24.621 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:24.621 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:24.621 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:24.621 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:24.621 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:24.621 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:24.621 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:24.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:24.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:24.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:24.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:24.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:24.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:24.622 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:24.622 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:24.622 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:24.622 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:24.623 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:24.623 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:24.623 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:24.623 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:24.623 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[98643]: Health check update: Degraded data redundancy: 60/630 objects degraded (9.524%), 18 pgs degraded (PG_DEGRADED) 2024-03-24T11:28:24.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:24.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:24.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:24.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:24.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:24.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:24.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:24.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:24.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:24 smithi062 ceph-mon[90553]: Health check update: Degraded data redundancy: 60/630 objects degraded (9.524%), 18 pgs degraded (PG_DEGRADED) 2024-03-24T11:28:25.401 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3[143820]: 2024-03-24T11:28:25.241+0000 7fb609cf47c0 -1 osd.3 0 read_superblock omap replica is missing. 2024-03-24T11:28:25.401 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3[143820]: 2024-03-24T11:28:25.289+0000 7fb609cf47c0 -1 osd.3 138 log_to_monitors true 2024-03-24T11:28:25.402 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:25.402 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:25.402 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:25.402 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:25.402 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:25.402 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:25.402 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:25.402 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:25.402 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:25.402 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:25.402 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:25.402 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:25.403 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:25.403 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:25.403 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:25.403 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:25.403 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:25.403 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:25.403 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:25.403 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:25.403 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:25.403 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:25.403 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:25.403 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:25.403 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:25.403 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:25.403 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:25.404 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:25.404 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:25.404 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:25.404 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:25.404 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:25.404 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:25.404 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:25.404 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:25.404 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:25.404 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:25.405 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:25.405 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:25.405 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:25.405 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:25.405 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:25.405 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:25.405 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:25.405 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:25.405 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:25.405 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:25.405 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:25.405 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:25.405 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:25.405 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:25.405 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:25.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:25.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:25.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:25.406 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:25.406 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:25.406 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:25.406 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:25.406 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:25.407 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:25.407 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:25.407 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:25.407 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:25.407 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:25.407 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:25.407 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:25.407 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:25.407 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:25.407 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:25.407 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:25.407 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:25.407 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:25.407 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:25.407 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:25.408 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:25.408 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:25.408 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:25.408 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:25.408 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:25.408 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:25.408 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:25.408 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:25.408 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:25.408 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:25.408 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:25.408 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:25.408 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:25.408 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:25.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:25.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:25.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:25.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:25.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:25.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:25.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:25.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:25.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:25.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:25.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:25.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:25.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:25.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:25.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:25.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:25.410 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:25.410 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:25.410 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:25.410 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:25.410 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:25.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:25.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:25.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:25.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:25.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:25.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:25.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:25.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:25.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:25.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:25.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:25.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:25.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:25.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:25.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:25.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:25.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:25.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:25.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:25.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:25.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:25.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:25.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:25.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:25.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:25.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:25.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:25.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:25.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:25.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:25.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:25.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:25.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:25.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:25.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:25.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:25.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:25.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:25.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:25.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:25.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:25.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:25.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:25.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:25.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:25.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:25.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:25.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:25.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:25.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:25.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:25.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:25.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:25.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:25.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:26.461 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3[143820]: 2024-03-24T11:28:26.328+0000 7fb5fb14c700 -1 osd.3 138 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T11:28:26.461 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:26.461 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: pgmap v179: 161 pgs: 23 active+undersized+degraded, 43 active+undersized, 95 active+clean; 585 KiB data, 176 MiB used, 715 GiB / 715 GiB avail; 80/630 objects degraded (12.698%) 2024-03-24T11:28:26.461 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:26.462 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: from='osd.3 [v2:172.21.15.62:6826/3208467027,v1:172.21.15.62:6827/3208467027]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-24T11:28:26.462 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-24T11:28:26.462 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:26.462 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:26.462 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:26.462 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:26.462 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:26.462 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:26.462 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:26.462 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:26.462 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:26.462 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:26.462 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:26.463 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:26.463 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:26.463 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:26.463 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:26.463 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:26.463 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:26.463 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:26.463 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:26.463 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:26.463 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:26.463 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:26.463 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:26.463 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:26.463 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:26.463 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:26.464 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:26.464 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:26.464 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:26.464 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:26.464 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:26.464 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:26.464 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:26.464 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:26.464 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:26.464 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:26.464 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:26.464 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:26.464 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:26.464 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:26.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:26.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 5 pgs inactive, 6 pgs peering) 2024-03-24T11:28:26.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:26.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:28:26.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:26.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:26.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:26.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:26.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:26.465 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:26.466 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: pgmap v179: 161 pgs: 23 active+undersized+degraded, 43 active+undersized, 95 active+clean; 585 KiB data, 176 MiB used, 715 GiB / 715 GiB avail; 80/630 objects degraded (12.698%) 2024-03-24T11:28:26.466 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:26.466 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: from='osd.3 [v2:172.21.15.62:6826/3208467027,v1:172.21.15.62:6827/3208467027]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-24T11:28:26.466 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-24T11:28:26.466 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:26.466 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:26.466 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:26.466 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:26.466 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:26.466 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:26.466 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:26.466 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:26.466 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:26.467 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:26.467 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:26.467 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:26.467 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:26.467 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:26.467 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:26.467 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:26.467 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:26.467 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:26.467 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:26.467 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:26.467 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:26.467 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:26.467 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:26.467 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:26.468 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:26.468 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:26.468 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:26.468 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:26.468 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:26.468 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:26.468 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:26.468 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:26.468 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:26.468 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:26.468 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:26.468 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:26.468 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:26.468 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:26.468 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:26.469 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:26.469 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:26.469 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 5 pgs inactive, 6 pgs peering) 2024-03-24T11:28:26.469 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:26.469 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:28:26.469 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:26.469 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:26.469 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:26.469 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:26.469 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:26.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:26.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: pgmap v179: 161 pgs: 23 active+undersized+degraded, 43 active+undersized, 95 active+clean; 585 KiB data, 176 MiB used, 715 GiB / 715 GiB avail; 80/630 objects degraded (12.698%) 2024-03-24T11:28:26.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:26.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: from='osd.3 [v2:172.21.15.62:6826/3208467027,v1:172.21.15.62:6827/3208467027]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-24T11:28:26.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-24T11:28:26.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:26.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:26.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:26.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:26.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:26.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:26.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:26.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:26.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:26.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:26.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:26.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:26.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:26.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:26.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:26.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:26.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:26.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:26.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:26.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:26.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:26.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:26.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:26.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:26.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:26.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:26.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:26.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:26.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:26.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:26.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:26.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:26.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:26.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:26.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:26.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:26.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:26.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:26.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:26.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:26.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:26.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 5 pgs inactive, 6 pgs peering) 2024-03-24T11:28:26.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:26.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:28:26.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:26.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:26.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:26.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:26.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:27.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:27.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:27.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-24T11:28:27.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: osdmap e141: 8 total, 7 up, 8 in 2024-03-24T11:28:27.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='osd.3 [v2:172.21.15.62:6826/3208467027,v1:172.21.15.62:6827/3208467027]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:28:27.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: failed to encode map e141 with expected crc 2024-03-24T11:28:27.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:28:27.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:27.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:27.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:27.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:27.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:27.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:27.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:27.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:27.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:27.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:27.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:27.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:27.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:27.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:27.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:27.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:27.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: failed to encode map e141 with expected crc 2024-03-24T11:28:27.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:27.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:27.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: failed to encode map e141 with expected crc 2024-03-24T11:28:27.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:27.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:27.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:27.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:27.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:27.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:27.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: failed to encode map e141 with expected crc 2024-03-24T11:28:27.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:27.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:27.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:27.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:27.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:27.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:27.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:27.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:27.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:27.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:27.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:27.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:27.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:27.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:27.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:27.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:27.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:27.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:27.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:27.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:27.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:27.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:27.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:27.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:27.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:27.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:27.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:27.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:27.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:27.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:27.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:27.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:27.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:27.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:27.626 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:27.626 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:27.626 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-24T11:28:27.626 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: osdmap e141: 8 total, 7 up, 8 in 2024-03-24T11:28:27.626 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='osd.3 [v2:172.21.15.62:6826/3208467027,v1:172.21.15.62:6827/3208467027]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:28:27.627 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: failed to encode map e141 with expected crc 2024-03-24T11:28:27.627 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:28:27.627 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:27.627 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:27.627 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:27.627 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:27.627 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:27.627 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:27.627 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:27.627 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:27.627 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:27.627 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:27.627 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:27.627 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:27.628 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:27.628 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:27.628 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:27.628 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:27.628 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: failed to encode map e141 with expected crc 2024-03-24T11:28:27.628 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:27.628 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:27.628 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: failed to encode map e141 with expected crc 2024-03-24T11:28:27.628 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:27.628 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:27.628 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:27.629 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:27.629 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-24T11:28:27.629 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: osdmap e141: 8 total, 7 up, 8 in 2024-03-24T11:28:27.629 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='osd.3 [v2:172.21.15.62:6826/3208467027,v1:172.21.15.62:6827/3208467027]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:28:27.629 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: failed to encode map e141 with expected crc 2024-03-24T11:28:27.629 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:28:27.629 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:27.629 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:27.629 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:27.629 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:27.629 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:27.629 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:27.629 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:27.629 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:27.630 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:27.630 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:27.630 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:27.630 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:27.630 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:27.630 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:27.630 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:27.630 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:27.630 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: failed to encode map e141 with expected crc 2024-03-24T11:28:27.630 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:27.630 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:27.630 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: failed to encode map e141 with expected crc 2024-03-24T11:28:27.630 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:27.630 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:27.631 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:27.631 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:27.631 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:27.631 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:27.631 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: failed to encode map e141 with expected crc 2024-03-24T11:28:27.631 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:27.631 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:27.631 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:27.631 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:27.631 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:27.632 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:27.632 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:27.632 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:27.632 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:27.632 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:27.632 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:27.632 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:27.632 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:27.632 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:27.632 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:27.632 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:27.632 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:27.632 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:27.632 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:27.632 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:27.633 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:27.633 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:27.633 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:27.633 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:27.633 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:27.633 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:27.633 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:27.633 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:27.633 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:27.633 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: failed to encode map e141 with expected crc 2024-03-24T11:28:27.633 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:27.633 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:27.634 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:27.634 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:27.634 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:27.634 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:27.634 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:27.634 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:27.634 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:27.634 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:27.634 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:27.634 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:27.635 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:27.635 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:27.635 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:27.635 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:27.635 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:27.635 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:27.635 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:27.635 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:27.635 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:27.635 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:27.635 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:27.635 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:27.635 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:27.636 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:27.636 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:27.636 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:27.636 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:27.636 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:27.636 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:27.636 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:27.636 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:27.636 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:27.637 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:27.637 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:27.637 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:27.637 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:27.637 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:27.637 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:27.637 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:27.637 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:27.637 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:28.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: failed to encode map e141 with expected crc 2024-03-24T11:28:28.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: pgmap v181: 161 pgs: 23 active+undersized+degraded, 43 active+undersized, 95 active+clean; 585 KiB data, 186 MiB used, 715 GiB / 715 GiB avail; 80/630 objects degraded (12.698%) 2024-03-24T11:28:28.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:28.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:28.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:28.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:28.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:28.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:28.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T11:28:28.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: failed to encode map e142 with expected crc 2024-03-24T11:28:28.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: failed to encode map e142 with expected crc 2024-03-24T11:28:28.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: failed to encode map e142 with expected crc 2024-03-24T11:28:28.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: osd.3 [v2:172.21.15.62:6826/3208467027,v1:172.21.15.62:6827/3208467027] boot 2024-03-24T11:28:28.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: osdmap e142: 8 total, 8 up, 8 in 2024-03-24T11:28:28.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T11:28:28.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: failed to encode map e142 with expected crc 2024-03-24T11:28:28.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:28.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:28.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:28.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:28.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:28.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:28.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:28.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:28.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:28.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:28.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:28.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:28.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:28.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:28.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:28.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:28.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:28.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:28.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:28.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:28.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:28.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:28.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:28.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:28.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:28.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:28.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:28.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:28.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:28.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:28.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:28.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:28.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:28.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:28.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:28.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:28.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:28.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:28.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:28.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:28.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:28.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:28.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:28.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:28.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:28.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:28.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:28.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:28.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:28.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:28.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:28.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:28.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:28.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:28.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:28.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:28.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:28.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:28.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:28.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: failed to encode map e141 with expected crc 2024-03-24T11:28:28.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: pgmap v181: 161 pgs: 23 active+undersized+degraded, 43 active+undersized, 95 active+clean; 585 KiB data, 186 MiB used, 715 GiB / 715 GiB avail; 80/630 objects degraded (12.698%) 2024-03-24T11:28:28.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:28.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:28.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:28.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:28.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:28.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:28.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T11:28:28.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: failed to encode map e142 with expected crc 2024-03-24T11:28:28.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: failed to encode map e142 with expected crc 2024-03-24T11:28:28.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: failed to encode map e142 with expected crc 2024-03-24T11:28:28.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: osd.3 [v2:172.21.15.62:6826/3208467027,v1:172.21.15.62:6827/3208467027] boot 2024-03-24T11:28:28.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: osdmap e142: 8 total, 8 up, 8 in 2024-03-24T11:28:28.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T11:28:28.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: failed to encode map e142 with expected crc 2024-03-24T11:28:28.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:28.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:28.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:28.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:28.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:28.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:28.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:28.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:28.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:28.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:28.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:28.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:28.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:28.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:28.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:28.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: failed to encode map e141 with expected crc 2024-03-24T11:28:28.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: pgmap v181: 161 pgs: 23 active+undersized+degraded, 43 active+undersized, 95 active+clean; 585 KiB data, 186 MiB used, 715 GiB / 715 GiB avail; 80/630 objects degraded (12.698%) 2024-03-24T11:28:28.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:28.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:28.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:28.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:28.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:28.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:28.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T11:28:28.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: failed to encode map e142 with expected crc 2024-03-24T11:28:28.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: failed to encode map e142 with expected crc 2024-03-24T11:28:28.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: failed to encode map e142 with expected crc 2024-03-24T11:28:28.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: osd.3 [v2:172.21.15.62:6826/3208467027,v1:172.21.15.62:6827/3208467027] boot 2024-03-24T11:28:28.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: osdmap e142: 8 total, 8 up, 8 in 2024-03-24T11:28:28.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T11:28:28.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: failed to encode map e142 with expected crc 2024-03-24T11:28:28.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:28.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:28.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:28.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:28.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:28.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:28.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:28.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:28.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:28.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:28.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:28.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:28.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:28.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:28.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:28.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:28.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:28.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:28.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:28.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:28.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:28.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:28.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:28.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:28.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:28.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:28.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:28.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:28.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:28.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:28.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:28.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:28.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:28.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:28.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:28.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:28.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:28.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:28.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:28.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:28.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:28.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:28.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:28.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:28.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:28.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:28.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:28.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:28.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:28.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:28.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:28.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:28.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:28.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:28.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:28.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:28.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:28.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:28.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:28.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:28.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:28.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:28.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:28.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:28.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:28.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:28.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:28.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:28.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:28.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:28.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:28.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:28.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:28.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:28.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:28.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:28.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:28.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:28.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:28.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:28.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:28.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:28.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:28.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:28.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:28.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:28.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:28.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:28.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:28.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:28.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:28.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:28.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:28.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:28.739 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:28.739 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:28.739 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:28.739 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:28.739 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:28.739 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:28.739 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:28.739 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:28.739 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:29.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:29.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:29.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:29.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: failed to encode map e143 with expected crc 2024-03-24T11:28:29.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: failed to encode map e143 with expected crc 2024-03-24T11:28:29.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: failed to encode map e143 with expected crc 2024-03-24T11:28:29.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: failed to encode map e143 with expected crc 2024-03-24T11:28:29.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: failed to encode map e143 with expected crc 2024-03-24T11:28:29.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: failed to encode map e143 with expected crc 2024-03-24T11:28:29.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: osdmap e143: 8 total, 8 up, 8 in 2024-03-24T11:28:29.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: failed to encode map e143 with expected crc 2024-03-24T11:28:29.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: failed to encode map e143 with expected crc 2024-03-24T11:28:29.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: failed to encode map e143 with expected crc 2024-03-24T11:28:29.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:29.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:29.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:29.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:29.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:29.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:29.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:29.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:29.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:29.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:29.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:29.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:29.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:29.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:29.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:29.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:29.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:29.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:29.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:29.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:29.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:29.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:29.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:29.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:29.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:29.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:29.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:29.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:29.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:29.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:29.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:29.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:29.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:29.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:29.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:29.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:29.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:29.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:29.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:29.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:29.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: Health check update: Degraded data redundancy: 80/630 objects degraded (12.698%), 23 pgs degraded (PG_DEGRADED) 2024-03-24T11:28:29.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:29.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:29.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:29.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:29.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:29.722 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:29.722 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:29.722 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:29.723 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: failed to encode map e143 with expected crc 2024-03-24T11:28:29.723 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: failed to encode map e143 with expected crc 2024-03-24T11:28:29.723 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: failed to encode map e143 with expected crc 2024-03-24T11:28:29.723 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: failed to encode map e143 with expected crc 2024-03-24T11:28:29.723 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: failed to encode map e143 with expected crc 2024-03-24T11:28:29.723 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: failed to encode map e143 with expected crc 2024-03-24T11:28:29.723 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: osdmap e143: 8 total, 8 up, 8 in 2024-03-24T11:28:29.723 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: failed to encode map e143 with expected crc 2024-03-24T11:28:29.723 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: failed to encode map e143 with expected crc 2024-03-24T11:28:29.723 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: failed to encode map e143 with expected crc 2024-03-24T11:28:29.723 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:29.723 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:29.723 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:29.723 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:29.724 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:29.724 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:29.724 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:29.724 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:29.724 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:29.724 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:29.724 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:29.724 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:29.724 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:29.724 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:29.724 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:29.724 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:29.724 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:29.724 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:29.724 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:29.725 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:29.725 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:29.725 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:29.725 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:29.725 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: failed to encode map e143 with expected crc 2024-03-24T11:28:29.725 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: failed to encode map e143 with expected crc 2024-03-24T11:28:29.725 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: failed to encode map e143 with expected crc 2024-03-24T11:28:29.725 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: failed to encode map e143 with expected crc 2024-03-24T11:28:29.725 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: failed to encode map e143 with expected crc 2024-03-24T11:28:29.725 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: failed to encode map e143 with expected crc 2024-03-24T11:28:29.726 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: osdmap e143: 8 total, 8 up, 8 in 2024-03-24T11:28:29.726 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: failed to encode map e143 with expected crc 2024-03-24T11:28:29.726 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: failed to encode map e143 with expected crc 2024-03-24T11:28:29.726 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: failed to encode map e143 with expected crc 2024-03-24T11:28:29.726 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:29.726 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:29.726 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:29.726 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:29.726 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:29.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:29.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:29.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:29.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:29.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:29.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:29.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:29.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:29.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:29.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:29.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:29.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:29.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:29.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:29.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:29.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:29.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:29.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:29.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:29.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:29.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:29.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:29.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:29.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:29.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:29.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:29.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:29.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:29.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:29.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:29.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:29.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:29.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:29.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:29.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:29.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: Health check update: Degraded data redundancy: 80/630 objects degraded (12.698%), 23 pgs degraded (PG_DEGRADED) 2024-03-24T11:28:29.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:29.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:29.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:29.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:29.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:29.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:29.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:29.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:29.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:29.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:29.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:29.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:29.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:29.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:29.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:29.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:29.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:29.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:29.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:29.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:29.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:29.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:29.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:29.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:29.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:29.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: Health check update: Degraded data redundancy: 80/630 objects degraded (12.698%), 23 pgs degraded (PG_DEGRADED) 2024-03-24T11:28:29.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:29.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:29.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:29.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:29.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:30.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:30.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: pgmap v184: 161 pgs: 7 peering, 17 active+undersized+degraded, 34 active+undersized, 103 active+clean; 585 KiB data, 190 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 54/630 objects degraded (8.571%) 2024-03-24T11:28:30.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:30.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:30.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:30.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:30.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:30.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:30.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:30.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:30.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:30.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:30.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:30.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:30.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:30.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:30.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:30.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:30.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:30.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:30.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:30.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:30.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:30.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:30.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:30.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:30.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:30.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:30.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:30.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:30.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: pgmap v184: 161 pgs: 7 peering, 17 active+undersized+degraded, 34 active+undersized, 103 active+clean; 585 KiB data, 190 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 54/630 objects degraded (8.571%) 2024-03-24T11:28:30.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:30.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:30.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:30.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:30.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:30.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:30.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:30.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:30.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:30.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:30.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:30.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:30.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:30.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:30.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:30.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:30.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:30.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:30.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:30.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:30.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:30.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:30.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:30.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:30.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:30.483 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:30.484 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:30.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:30.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:30.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:30.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:30.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:30.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:30.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:30.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:30.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:30.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:30.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:30.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:30.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:30.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:30.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:30.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:30.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:30.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:30.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:30.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:30.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:30.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:30.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:30.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:30.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:30.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:30.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:30.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:30.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:30.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:30.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:30.486 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:30.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:30.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:30.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:30.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:30.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:30.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:30.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:30.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:30.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:30.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:30.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:30.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:30.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:30.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:30.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:30.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:30.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:30.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:30.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:30.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:30.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:30.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:30.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:30.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:30.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:30.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:30.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:30.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:30.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:30.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:30.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:30.489 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:30.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:30.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: pgmap v184: 161 pgs: 7 peering, 17 active+undersized+degraded, 34 active+undersized, 103 active+clean; 585 KiB data, 190 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 54/630 objects degraded (8.571%) 2024-03-24T11:28:30.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:30.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:30.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:30.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:30.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:30.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:30.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:30.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:30.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:30.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:30.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:30.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:30.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:30.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:30.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:30.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:30.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:30.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:30.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:30.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:30.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:30.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:30.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:30.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:30.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:30.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:30.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:30.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:30.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:30.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:30.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:30.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:30.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:30.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:30.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:30.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:30.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:30.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:30.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:30.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:30.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:30.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:30.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:30.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:30.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:30.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:30.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:30.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:30.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:30.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:30.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:30.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:30.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:30.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:30.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:30.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:30.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:30.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:30.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:30.977 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:28:30 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:28:30] "GET /metrics HTTP/1.1" 200 38514 "" "Prometheus/2.33.4" 2024-03-24T11:28:31.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:31.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:31.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:31.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:31.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:31.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:31.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:31.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:31.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:31.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:31.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:31.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:31.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:31.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:31.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:31.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:31.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:31.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:31.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:31.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:31.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:31.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:31.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:31.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:31.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:31.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:31.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:31.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:31.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:31.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:31.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:31.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:31.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:31.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:31.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:31.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:31.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:31.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:31.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:31.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:31.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:31.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:31.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:31.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:31.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:31.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:31.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:31.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:31.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:31.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:31.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:31.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:31.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:31.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:31.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:31.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:31.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:31.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:31.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:31.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:31.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:31.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:31.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:31.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:31.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:31.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:31.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:31.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:31.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:31.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:31.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:31.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:31.614 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:31.614 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:31.614 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:31.615 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:31.615 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:31.615 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:31.615 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:31.615 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:31.615 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:31.615 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:31.615 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:31.615 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:31.615 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:31.615 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:31.615 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:31.615 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:31.615 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:31.616 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:31.616 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:31.616 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:31.616 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:31.616 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:31.616 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:31.616 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:31.616 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:31.616 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:31.617 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:31.617 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:31.617 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:31.617 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:31.617 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:31.617 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:31.617 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:31.617 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:31.617 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:31.617 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:31.617 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:31.618 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:31.618 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:31.618 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:31.618 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:31.618 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:31.618 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:31.618 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:31.618 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:31.618 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:31.618 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:31.618 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:31.618 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:31.618 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:31.618 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:31.619 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:31.619 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:31.619 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:31.619 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:31.619 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:31.619 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:31.619 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:31.619 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:31.619 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:31.619 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:31.619 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:31.620 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:31.620 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:31.620 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:31.620 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:31.620 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:31.620 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:31.620 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:31.620 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:31.620 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:31.620 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:31.620 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:31.620 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:31.620 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:31.620 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:31.620 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:31.621 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:31.621 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:31.621 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:31.621 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:31.621 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:31.621 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:31.621 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:31.621 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:31.621 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:31.621 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:31.621 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:31.621 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:31.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:31.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:31.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:31.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:31.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:31.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:31.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:31.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:31.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:31.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:31.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:31.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:31.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:31.622 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:31.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:31.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:31.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:31.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:31.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:31.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:31.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:31.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:31.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:31.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:31.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:31.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:31.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:31.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:31.623 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:31.624 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:31.624 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:31.624 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:31.624 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:31.624 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:31.624 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:31.625 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:31.625 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:31.625 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:31.625 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:31.625 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:31.625 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:31.625 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:31.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:31.626 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:31.626 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:31.626 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:31.626 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:31.626 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:31.626 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:31.626 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:31.626 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:31.627 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:31.627 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:31.627 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:31.627 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:31.977 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:28:31 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:28:31.611+0000 7f4e4aad7700 -1 mgr.server reply reply (16) Device or resource busy unsafe to stop osd(s) at this time (7 PGs are or would become offline) 2024-03-24T11:28:32.447 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[90553]: pgmap v185: 161 pgs: 7 peering, 17 active+undersized+degraded, 34 active+undersized, 103 active+clean; 585 KiB data, 190 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 54/630 objects degraded (8.571%) 2024-03-24T11:28:32.447 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:32.447 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:32.447 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:32.448 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:32.448 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:32.448 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:32.448 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:32.448 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:32.448 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:32.448 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:32.448 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:32.448 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:32.448 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:32.448 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:32.448 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:32.448 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:32.448 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:32.448 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:32.449 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:32.449 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:32.449 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:32.449 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:32.449 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-24T11:28:32.449 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[90553]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-24T11:28:32.449 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[90553]: Upgrade: unsafe to stop osd(s) at this time (7 PGs are or would become offline) 2024-03-24T11:28:32.449 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[98643]: pgmap v185: 161 pgs: 7 peering, 17 active+undersized+degraded, 34 active+undersized, 103 active+clean; 585 KiB data, 190 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 54/630 objects degraded (8.571%) 2024-03-24T11:28:32.449 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:32.449 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:32.449 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:32.450 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:32.450 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:32.450 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:32.450 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:32.450 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:32.450 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:32.450 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:32.450 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:32.450 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:32.450 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:32.450 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:32.450 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:32.450 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:32.450 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:32.451 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:32.451 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:32.451 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:32.451 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:32.451 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:32.451 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-24T11:28:32.451 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[98643]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-24T11:28:32.451 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:32 smithi062 ceph-mon[98643]: Upgrade: unsafe to stop osd(s) at this time (7 PGs are or would become offline) 2024-03-24T11:28:32.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:32 smithi139 ceph-mon[91894]: pgmap v185: 161 pgs: 7 peering, 17 active+undersized+degraded, 34 active+undersized, 103 active+clean; 585 KiB data, 190 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 54/630 objects degraded (8.571%) 2024-03-24T11:28:32.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:32 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:32.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:32 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:32.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:32 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:32.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:32.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:32.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:32.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:32.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:32.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:32 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:32.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:32.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:32.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:32.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:28:32.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:32.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:32 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:28:32.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:32.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:32.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:32.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:28:32.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:32.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:32.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:32.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-24T11:28:32.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:32 smithi139 ceph-mon[91894]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-24T11:28:32.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:32 smithi139 ceph-mon[91894]: Upgrade: unsafe to stop osd(s) at this time (7 PGs are or would become offline) 2024-03-24T11:28:33.339 INFO:teuthology.orchestra.run.smithi062.stdout:true 2024-03-24T11:28:33.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:33 smithi139 ceph-mon[91894]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:28:33.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:33 smithi062 ceph-mon[98643]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:28:33.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:33 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:28:34.056 INFO:teuthology.orchestra.run.smithi062.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T11:28:34.056 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi062 smithi062 running 2s ago 18m - - 2024-03-24T11:28:34.056 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi139 smithi139 running 14s ago 17m - - 2024-03-24T11:28:34.056 INFO:teuthology.orchestra.run.smithi062.stdout:alertmanager.a smithi062 *:9093,9094 running (7m) 2s ago 12m 24.7M - 0.23.0 ba2b418f427c 818298d58140 2024-03-24T11:28:34.057 INFO:teuthology.orchestra.run.smithi062.stdout:grafana.a smithi139 *:3000 running (7m) 14s ago 12m 53.5M - 8.3.5 dad864ee21e9 b48167e7171e 2024-03-24T11:28:34.057 INFO:teuthology.orchestra.run.smithi062.stdout:iscsi.foo.smithi062.dggsvk smithi062 running (6m) 2s ago 11m 60.3M - 3.5 e1d6a67b021e 0a98806ec839 2024-03-24T11:28:34.057 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.x smithi139 *:8443,9283,8765 running (5m) 14s ago 17m 451M - 19.0.0-1667-gdb0330b1 4f325f8440e9 d94987b11387 2024-03-24T11:28:34.057 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.y smithi062 *:8443,9283,8765 running (6m) 2s ago 18m 526M - 19.0.0-1667-gdb0330b1 4f325f8440e9 ca1a49028d65 2024-03-24T11:28:34.057 INFO:teuthology.orchestra.run.smithi062.stdout:mon.a smithi062 running (4m) 2s ago 19m 66.8M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 f7afe6e28c88 2024-03-24T11:28:34.057 INFO:teuthology.orchestra.run.smithi062.stdout:mon.b smithi139 running (3m) 14s ago 17m 44.2M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 ca357beea4dd 2024-03-24T11:28:34.057 INFO:teuthology.orchestra.run.smithi062.stdout:mon.c smithi062 running (4m) 2s ago 17m 54.2M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 459e7449ec39 2024-03-24T11:28:34.057 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.a smithi062 *:9100 running (8m) 2s ago 13m 17.5M - 1.3.1 1dbe0e931976 79d5a02a999f 2024-03-24T11:28:34.057 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.b smithi139 *:9100 running (7m) 14s ago 13m 15.8M - 1.3.1 1dbe0e931976 8f9f4608e465 2024-03-24T11:28:34.057 INFO:teuthology.orchestra.run.smithi062.stdout:osd.0 smithi062 running (2m) 2s ago 16m 55.3M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 dfe4920b49c4 2024-03-24T11:28:34.057 INFO:teuthology.orchestra.run.smithi062.stdout:osd.1 smithi062 running (100s) 2s ago 16m 45.7M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 c443b3fe97c1 2024-03-24T11:28:34.057 INFO:teuthology.orchestra.run.smithi062.stdout:osd.2 smithi062 running (37s) 2s ago 16m 43.1M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 0eba40dc96b3 2024-03-24T11:28:34.057 INFO:teuthology.orchestra.run.smithi062.stdout:osd.3 smithi062 running (15m) 2s ago 15m 62.9M 4096M 17.2.0 e1d6a67b021e 1d2868bd5195 2024-03-24T11:28:34.058 INFO:teuthology.orchestra.run.smithi062.stdout:osd.4 smithi139 running (15m) 14s ago 15m 66.6M 4096M 17.2.0 e1d6a67b021e 7568bfa9e74d 2024-03-24T11:28:34.058 INFO:teuthology.orchestra.run.smithi062.stdout:osd.5 smithi139 running (15m) 14s ago 15m 60.6M 4096M 17.2.0 e1d6a67b021e dd0abd599a14 2024-03-24T11:28:34.058 INFO:teuthology.orchestra.run.smithi062.stdout:osd.6 smithi139 running (14m) 14s ago 14m 58.9M 4096M 17.2.0 e1d6a67b021e a4b1289aca69 2024-03-24T11:28:34.058 INFO:teuthology.orchestra.run.smithi062.stdout:osd.7 smithi139 running (14m) 14s ago 14m 63.1M 4096M 17.2.0 e1d6a67b021e bc7fadbc1384 2024-03-24T11:28:34.058 INFO:teuthology.orchestra.run.smithi062.stdout:prometheus.a smithi139 *:9095 running (6m) 14s ago 13m 71.2M - 2.33.4 514e6a882f6e 2ed47f810d33 2024-03-24T11:28:34.058 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi062.sacpds smithi062 *:8000 running (11m) 2s ago 11m 74.3M - 17.2.0 e1d6a67b021e 89f4ac308e98 2024-03-24T11:28:34.058 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi139.yljesh smithi139 *:8000 running (11m) 14s ago 11m 73.4M - 17.2.0 e1d6a67b021e e189bdc12cae 2024-03-24T11:28:34.058 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi062.xvpxnk smithi062 *:80 running (11m) 2s ago 11m 74.2M - 17.2.0 e1d6a67b021e d1aa74adab44 2024-03-24T11:28:34.058 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi139.pbtrmp smithi139 *:80 running (11m) 14s ago 11m 77.1M - 17.2.0 e1d6a67b021e 7f8e336755d7 2024-03-24T11:28:34.426 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:34 smithi062 ceph-mon[98643]: pgmap v186: 161 pgs: 7 peering, 154 active+clean; 585 KiB data, 191 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:28:34.426 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:34 smithi062 ceph-mon[98643]: from='client.44427 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:28:34.426 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:34 smithi062 ceph-mon[98643]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 54/630 objects degraded (8.571%), 17 pgs degraded) 2024-03-24T11:28:34.426 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:34 smithi062 ceph-mon[98643]: Cluster is now healthy 2024-03-24T11:28:34.426 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:34 smithi062 ceph-mon[98643]: from='client.44433 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:28:34.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:34 smithi062 ceph-mon[90553]: pgmap v186: 161 pgs: 7 peering, 154 active+clean; 585 KiB data, 191 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:28:34.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:34 smithi062 ceph-mon[90553]: from='client.44427 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:28:34.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:34 smithi062 ceph-mon[90553]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 54/630 objects degraded (8.571%), 17 pgs degraded) 2024-03-24T11:28:34.427 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:34 smithi062 ceph-mon[90553]: Cluster is now healthy 2024-03-24T11:28:34.427 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:34 smithi062 ceph-mon[90553]: from='client.44433 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:28:34.482 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:28:34.482 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": { 2024-03-24T11:28:34.482 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 3 2024-03-24T11:28:34.482 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:28:34.482 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": { 2024-03-24T11:28:34.482 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-24T11:28:34.482 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:28:34.482 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": { 2024-03-24T11:28:34.482 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4, 2024-03-24T11:28:34.482 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 4 2024-03-24T11:28:34.483 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:28:34.483 INFO:teuthology.orchestra.run.smithi062.stdout: "rgw": { 2024-03-24T11:28:34.483 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4 2024-03-24T11:28:34.483 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:28:34.483 INFO:teuthology.orchestra.run.smithi062.stdout: "overall": { 2024-03-24T11:28:34.483 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 8, 2024-03-24T11:28:34.483 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 9 2024-03-24T11:28:34.483 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-24T11:28:34.483 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:28:34.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:34 smithi139 ceph-mon[91894]: pgmap v186: 161 pgs: 7 peering, 154 active+clean; 585 KiB data, 191 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:28:34.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:34 smithi139 ceph-mon[91894]: from='client.44427 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:28:34.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:34 smithi139 ceph-mon[91894]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 54/630 objects degraded (8.571%), 17 pgs degraded) 2024-03-24T11:28:34.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:34 smithi139 ceph-mon[91894]: Cluster is now healthy 2024-03-24T11:28:34.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:34 smithi139 ceph-mon[91894]: from='client.44433 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:28:34.831 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:28:34.831 INFO:teuthology.orchestra.run.smithi062.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69", 2024-03-24T11:28:34.831 INFO:teuthology.orchestra.run.smithi062.stdout: "in_progress": true, 2024-03-24T11:28:34.831 INFO:teuthology.orchestra.run.smithi062.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T11:28:34.831 INFO:teuthology.orchestra.run.smithi062.stdout: "services_complete": [ 2024-03-24T11:28:34.831 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr", 2024-03-24T11:28:34.831 INFO:teuthology.orchestra.run.smithi062.stdout: "mon" 2024-03-24T11:28:34.831 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-03-24T11:28:34.831 INFO:teuthology.orchestra.run.smithi062.stdout: "progress": "9/23 daemons upgraded", 2024-03-24T11:28:34.831 INFO:teuthology.orchestra.run.smithi062.stdout: "message": "Currently upgrading osd daemons", 2024-03-24T11:28:34.832 INFO:teuthology.orchestra.run.smithi062.stdout: "is_paused": false 2024-03-24T11:28:34.832 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:28:35.243 INFO:teuthology.orchestra.run.smithi062.stdout:HEALTH_OK 2024-03-24T11:28:35.641 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:35 smithi062 ceph-mon[98643]: from='client.44439 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:28:35.641 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:35.641 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:35.642 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:35.642 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:35 smithi062 ceph-mon[98643]: from='client.? 172.21.15.62:0/2390826789' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:35.642 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:35 smithi062 ceph-mon[98643]: from='client.44451 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:28:35.642 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:35 smithi062 ceph-mon[98643]: from='client.? 172.21.15.62:0/4107195026' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:28:35.642 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:35 smithi062 ceph-mon[90553]: from='client.44439 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:28:35.642 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:35.642 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:35.642 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:35.642 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:35 smithi062 ceph-mon[90553]: from='client.? 172.21.15.62:0/2390826789' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:35.643 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:35 smithi062 ceph-mon[90553]: from='client.44451 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:28:35.643 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:35 smithi062 ceph-mon[90553]: from='client.? 172.21.15.62:0/4107195026' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:28:35.754 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:35 smithi139 ceph-mon[91894]: from='client.44439 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:28:35.755 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:35.755 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:35.755 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:35.755 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:35 smithi139 ceph-mon[91894]: from='client.? 172.21.15.62:0/2390826789' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:28:35.755 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:35 smithi139 ceph-mon[91894]: from='client.44451 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:28:35.755 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:35 smithi139 ceph-mon[91894]: from='client.? 172.21.15.62:0/4107195026' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:28:36.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:36 smithi062 ceph-mon[90553]: pgmap v187: 161 pgs: 161 active+clean; 585 KiB data, 191 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:28:36.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:36.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:36.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:36.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:36 smithi062 ceph-mon[98643]: pgmap v187: 161 pgs: 161 active+clean; 585 KiB data, 191 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:28:36.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:36.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:36.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:36.799 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:36 smithi139 ceph-mon[91894]: pgmap v187: 161 pgs: 161 active+clean; 585 KiB data, 191 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:28:36.799 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:36.799 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:36.799 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:38.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:38 smithi062 ceph-mon[90553]: pgmap v188: 161 pgs: 161 active+clean; 585 KiB data, 191 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:28:38.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:38 smithi062 ceph-mon[98643]: pgmap v188: 161 pgs: 161 active+clean; 585 KiB data, 191 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:28:38.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:38 smithi139 ceph-mon[91894]: pgmap v188: 161 pgs: 161 active+clean; 585 KiB data, 191 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:28:41.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:28:40 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:28:40] "GET /metrics HTTP/1.1" 200 38514 "" "Prometheus/2.33.4" 2024-03-24T11:28:41.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:41 smithi062 ceph-mon[90553]: pgmap v189: 161 pgs: 161 active+clean; 585 KiB data, 191 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:28:41.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:41.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:41.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:41.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:41 smithi062 ceph-mon[98643]: pgmap v189: 161 pgs: 161 active+clean; 585 KiB data, 191 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:28:41.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:41.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:41.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:41.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:41 smithi139 ceph-mon[91894]: pgmap v189: 161 pgs: 161 active+clean; 585 KiB data, 191 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:28:41.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:41.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:41.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:42.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:42 smithi062 ceph-mon[90553]: pgmap v190: 161 pgs: 161 active+clean; 585 KiB data, 191 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:28:42.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:42.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:28:42.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:42.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:42 smithi062 ceph-mon[98643]: pgmap v190: 161 pgs: 161 active+clean; 585 KiB data, 191 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:28:42.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:42.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:28:42.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:42.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:42 smithi139 ceph-mon[91894]: pgmap v190: 161 pgs: 161 active+clean; 585 KiB data, 191 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:28:42.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:42.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:28:42.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:43.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:43 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:28:43.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:43 smithi062 ceph-mon[98643]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:28:43.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:43 smithi139 ceph-mon[91894]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:28:44.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:44 smithi062 ceph-mon[90553]: pgmap v191: 161 pgs: 161 active+clean; 585 KiB data, 191 MiB used, 715 GiB / 715 GiB avail; 341 B/s rd, 0 op/s 2024-03-24T11:28:44.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:44 smithi062 ceph-mon[98643]: pgmap v191: 161 pgs: 161 active+clean; 585 KiB data, 191 MiB used, 715 GiB / 715 GiB avail; 341 B/s rd, 0 op/s 2024-03-24T11:28:44.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:44 smithi139 ceph-mon[91894]: pgmap v191: 161 pgs: 161 active+clean; 585 KiB data, 191 MiB used, 715 GiB / 715 GiB avail; 341 B/s rd, 0 op/s 2024-03-24T11:28:46.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:46 smithi062 ceph-mon[90553]: pgmap v192: 161 pgs: 161 active+clean; 585 KiB data, 191 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:28:46.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:46 smithi062 ceph-mon[98643]: pgmap v192: 161 pgs: 161 active+clean; 585 KiB data, 191 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:28:46.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:46 smithi139 ceph-mon[91894]: pgmap v192: 161 pgs: 161 active+clean; 585 KiB data, 191 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:28:47.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-24T11:28:47.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:47 smithi062 ceph-mon[98643]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-24T11:28:47.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:47 smithi062 ceph-mon[98643]: Upgrade: osd.3 is safe to restart 2024-03-24T11:28:47.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-24T11:28:47.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:47 smithi062 ceph-mon[90553]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-24T11:28:47.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:47 smithi062 ceph-mon[90553]: Upgrade: osd.3 is safe to restart 2024-03-24T11:28:47.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-24T11:28:47.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:47 smithi139 ceph-mon[91894]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-24T11:28:47.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:47 smithi139 ceph-mon[91894]: Upgrade: osd.3 is safe to restart 2024-03-24T11:28:48.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:48 smithi062 ceph-mon[90553]: pgmap v193: 161 pgs: 161 active+clean; 585 KiB data, 191 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:28:48.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:48 smithi062 ceph-mon[98643]: pgmap v193: 161 pgs: 161 active+clean; 585 KiB data, 191 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:28:48.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:48 smithi139 ceph-mon[91894]: pgmap v193: 161 pgs: 161 active+clean; 585 KiB data, 191 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:28:49.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:49 smithi062 ceph-mon[90553]: Upgrade: Updating osd.3 2024-03-24T11:28:49.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-24T11:28:49.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:49.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:49 smithi062 ceph-mon[90553]: Deploying daemon osd.3 on smithi062 2024-03-24T11:28:49.359 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:49 smithi062 ceph-mon[98643]: Upgrade: Updating osd.3 2024-03-24T11:28:49.359 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-24T11:28:49.359 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:49.359 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:49 smithi062 ceph-mon[98643]: Deploying daemon osd.3 on smithi062 2024-03-24T11:28:49.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:49 smithi139 ceph-mon[91894]: Upgrade: Updating osd.3 2024-03-24T11:28:49.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-24T11:28:49.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:28:49.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:49 smithi139 ceph-mon[91894]: Deploying daemon osd.3 on smithi062 2024-03-24T11:28:50.383 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:50 smithi062 ceph-mon[90553]: pgmap v194: 161 pgs: 161 active+clean; 585 KiB data, 191 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:28:50.383 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:50 smithi062 ceph-mon[98643]: pgmap v194: 161 pgs: 161 active+clean; 585 KiB data, 191 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:28:50.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:50 smithi139 ceph-mon[91894]: pgmap v194: 161 pgs: 161 active+clean; 585 KiB data, 191 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:28:50.977 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:28:50 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:28:50] "GET /metrics HTTP/1.1" 200 38522 "" "Prometheus/2.33.4" 2024-03-24T11:28:51.603 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:51 smithi062 systemd[1]: Stopping Ceph osd.3 for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:28:51.977 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:51 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3[143820]: 2024-03-24T11:28:51.602+0000 7fb60035b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T11:28:51.977 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:51 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3[143820]: 2024-03-24T11:28:51.602+0000 7fb60035b700 -1 osd.3 143 *** Got signal Terminated *** 2024-03-24T11:28:51.977 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:51 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3[143820]: 2024-03-24T11:28:51.602+0000 7fb60035b700 -1 osd.3 143 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T11:28:52.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:52 smithi062 ceph-mon[90553]: pgmap v195: 161 pgs: 161 active+clean; 585 KiB data, 191 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:28:52.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:52 smithi062 ceph-mon[90553]: osd.3 marked itself down and dead 2024-03-24T11:28:52.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:52 smithi062 ceph-mon[98643]: pgmap v195: 161 pgs: 161 active+clean; 585 KiB data, 191 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:28:52.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:52 smithi062 ceph-mon[98643]: osd.3 marked itself down and dead 2024-03-24T11:28:52.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:52 smithi139 ceph-mon[91894]: pgmap v195: 161 pgs: 161 active+clean; 585 KiB data, 191 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:28:52.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:52 smithi139 ceph-mon[91894]: osd.3 marked itself down and dead 2024-03-24T11:28:52.907 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:52 smithi062 podman[148921]: 2024-03-24 11:28:52.619948198 +0000 UTC m=+1.143799546 container died 4c055d2ab5c11100ccf0829ef77e12e9095c9a5038dff3584f90d694587a6c45 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-db0330b) 2024-03-24T11:28:53.172 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:53 smithi062 podman[148921]: 2024-03-24 11:28:53.080415719 +0000 UTC m=+1.604267103 container cleanup 4c055d2ab5c11100ccf0829ef77e12e9095c9a5038dff3584f90d694587a6c45 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3, org.label-schema.build-date=20240307, RELEASE=squid-db0330b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T11:28:53.173 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:53 smithi062 bash[148921]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3 2024-03-24T11:28:53.462 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:53 smithi062 podman[149093]: 2024-03-24 11:28:53.273733168 +0000 UTC m=+0.726998344 container remove 4c055d2ab5c11100ccf0829ef77e12e9095c9a5038dff3584f90d694587a6c45 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240307, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-24T11:28:53.463 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:53 smithi062 ceph-mon[98643]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T11:28:53.463 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:53 smithi062 ceph-mon[98643]: failed to encode map e144 with expected crc 2024-03-24T11:28:53.463 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:53 smithi062 ceph-mon[98643]: failed to encode map e144 with expected crc 2024-03-24T11:28:53.463 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:53 smithi062 ceph-mon[98643]: osdmap e144: 8 total, 7 up, 8 in 2024-03-24T11:28:53.463 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:53 smithi062 ceph-mon[98643]: failed to encode map e144 with expected crc 2024-03-24T11:28:53.463 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:53 smithi062 ceph-mon[98643]: failed to encode map e144 with expected crc 2024-03-24T11:28:53.463 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:53 smithi062 ceph-mon[98643]: failed to encode map e144 with expected crc 2024-03-24T11:28:53.463 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:53 smithi062 ceph-mon[98643]: failed to encode map e144 with expected crc 2024-03-24T11:28:53.463 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:53 smithi062 ceph-mon[98643]: failed to encode map e144 with expected crc 2024-03-24T11:28:53.463 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:53 smithi062 ceph-mon[98643]: failed to encode map e144 with expected crc 2024-03-24T11:28:53.463 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:53 smithi062 ceph-mon[98643]: failed to encode map e144 with expected crc 2024-03-24T11:28:53.464 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:53 smithi062 ceph-mon[98643]: failed to encode map e144 with expected crc 2024-03-24T11:28:53.464 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:53 smithi062 ceph-mon[98643]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:28:53.464 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:53 smithi062 ceph-mon[90553]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T11:28:53.464 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:53 smithi062 ceph-mon[90553]: failed to encode map e144 with expected crc 2024-03-24T11:28:53.464 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:53 smithi062 ceph-mon[90553]: failed to encode map e144 with expected crc 2024-03-24T11:28:53.464 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:53 smithi062 ceph-mon[90553]: osdmap e144: 8 total, 7 up, 8 in 2024-03-24T11:28:53.464 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:53 smithi062 ceph-mon[90553]: failed to encode map e144 with expected crc 2024-03-24T11:28:53.464 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:53 smithi062 ceph-mon[90553]: failed to encode map e144 with expected crc 2024-03-24T11:28:53.464 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:53 smithi062 ceph-mon[90553]: failed to encode map e144 with expected crc 2024-03-24T11:28:53.464 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:53 smithi062 ceph-mon[90553]: failed to encode map e144 with expected crc 2024-03-24T11:28:53.464 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:53 smithi062 ceph-mon[90553]: failed to encode map e144 with expected crc 2024-03-24T11:28:53.464 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:53 smithi062 ceph-mon[90553]: failed to encode map e144 with expected crc 2024-03-24T11:28:53.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:53 smithi062 ceph-mon[90553]: failed to encode map e144 with expected crc 2024-03-24T11:28:53.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:53 smithi062 ceph-mon[90553]: failed to encode map e144 with expected crc 2024-03-24T11:28:53.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:53 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:28:53.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:53 smithi139 ceph-mon[91894]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T11:28:53.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:53 smithi139 ceph-mon[91894]: failed to encode map e144 with expected crc 2024-03-24T11:28:53.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:53 smithi139 ceph-mon[91894]: failed to encode map e144 with expected crc 2024-03-24T11:28:53.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:53 smithi139 ceph-mon[91894]: osdmap e144: 8 total, 7 up, 8 in 2024-03-24T11:28:53.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:53 smithi139 ceph-mon[91894]: failed to encode map e144 with expected crc 2024-03-24T11:28:53.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:53 smithi139 ceph-mon[91894]: failed to encode map e144 with expected crc 2024-03-24T11:28:53.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:53 smithi139 ceph-mon[91894]: failed to encode map e144 with expected crc 2024-03-24T11:28:53.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:53 smithi139 ceph-mon[91894]: failed to encode map e144 with expected crc 2024-03-24T11:28:53.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:53 smithi139 ceph-mon[91894]: failed to encode map e144 with expected crc 2024-03-24T11:28:53.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:53 smithi139 ceph-mon[91894]: failed to encode map e144 with expected crc 2024-03-24T11:28:53.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:53 smithi139 ceph-mon[91894]: failed to encode map e144 with expected crc 2024-03-24T11:28:53.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:53 smithi139 ceph-mon[91894]: failed to encode map e144 with expected crc 2024-03-24T11:28:53.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:53 smithi139 ceph-mon[91894]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:28:54.404 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:54 smithi062 podman[149251]: 2024-03-24 11:28:54.023125703 +0000 UTC m=+0.019572566 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:28:54.405 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:54 smithi062 podman[149251]: 2024-03-24 11:28:54.143376905 +0000 UTC m=+0.139823765 container create 05e9b6a05b76fb14d9616d5381d26c066522a8a6ce705d1152df6c84ebba2e21 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, ceph=True, org.label-schema.license=GPLv2) 2024-03-24T11:28:54.405 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:54 smithi062 ceph-mon[98643]: pgmap v197: 161 pgs: 27 stale+active+clean, 134 active+clean; 585 KiB data, 191 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T11:28:54.405 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:54 smithi062 ceph-mon[98643]: failed to encode map e145 with expected crc 2024-03-24T11:28:54.405 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:54 smithi062 ceph-mon[98643]: failed to encode map e145 with expected crc 2024-03-24T11:28:54.405 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:54 smithi062 ceph-mon[98643]: osdmap e145: 8 total, 7 up, 8 in 2024-03-24T11:28:54.405 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:54 smithi062 ceph-mon[98643]: failed to encode map e145 with expected crc 2024-03-24T11:28:54.405 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:54 smithi062 ceph-mon[98643]: failed to encode map e145 with expected crc 2024-03-24T11:28:54.405 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:54 smithi062 ceph-mon[98643]: failed to encode map e145 with expected crc 2024-03-24T11:28:54.406 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:54 smithi062 ceph-mon[98643]: failed to encode map e145 with expected crc 2024-03-24T11:28:54.406 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:54 smithi062 ceph-mon[98643]: failed to encode map e145 with expected crc 2024-03-24T11:28:54.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:54 smithi062 ceph-mon[90553]: pgmap v197: 161 pgs: 27 stale+active+clean, 134 active+clean; 585 KiB data, 191 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T11:28:54.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:54 smithi062 ceph-mon[90553]: failed to encode map e145 with expected crc 2024-03-24T11:28:54.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:54 smithi062 ceph-mon[90553]: failed to encode map e145 with expected crc 2024-03-24T11:28:54.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:54 smithi062 ceph-mon[90553]: osdmap e145: 8 total, 7 up, 8 in 2024-03-24T11:28:54.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:54 smithi062 ceph-mon[90553]: failed to encode map e145 with expected crc 2024-03-24T11:28:54.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:54 smithi062 ceph-mon[90553]: failed to encode map e145 with expected crc 2024-03-24T11:28:54.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:54 smithi062 ceph-mon[90553]: failed to encode map e145 with expected crc 2024-03-24T11:28:54.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:54 smithi062 ceph-mon[90553]: failed to encode map e145 with expected crc 2024-03-24T11:28:54.407 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:54 smithi062 ceph-mon[90553]: failed to encode map e145 with expected crc 2024-03-24T11:28:54.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:54 smithi139 ceph-mon[91894]: pgmap v197: 161 pgs: 27 stale+active+clean, 134 active+clean; 585 KiB data, 191 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T11:28:54.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:54 smithi139 ceph-mon[91894]: failed to encode map e145 with expected crc 2024-03-24T11:28:54.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:54 smithi139 ceph-mon[91894]: failed to encode map e145 with expected crc 2024-03-24T11:28:54.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:54 smithi139 ceph-mon[91894]: osdmap e145: 8 total, 7 up, 8 in 2024-03-24T11:28:54.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:54 smithi139 ceph-mon[91894]: failed to encode map e145 with expected crc 2024-03-24T11:28:54.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:54 smithi139 ceph-mon[91894]: failed to encode map e145 with expected crc 2024-03-24T11:28:54.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:54 smithi139 ceph-mon[91894]: failed to encode map e145 with expected crc 2024-03-24T11:28:54.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:54 smithi139 ceph-mon[91894]: failed to encode map e145 with expected crc 2024-03-24T11:28:54.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:54 smithi139 ceph-mon[91894]: failed to encode map e145 with expected crc 2024-03-24T11:28:54.686 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:54 smithi062 podman[149251]: 2024-03-24 11:28:54.404785823 +0000 UTC m=+0.401232694 container init 05e9b6a05b76fb14d9616d5381d26c066522a8a6ce705d1152df6c84ebba2e21 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-24T11:28:54.687 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:54 smithi062 podman[149251]: 2024-03-24 11:28:54.408211085 +0000 UTC m=+0.404657954 container start 05e9b6a05b76fb14d9616d5381d26c066522a8a6ce705d1152df6c84ebba2e21 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, RELEASE=squid-db0330b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-24T11:28:54.687 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:54 smithi062 podman[149251]: 2024-03-24 11:28:54.488051021 +0000 UTC m=+0.484497882 container attach 05e9b6a05b76fb14d9616d5381d26c066522a8a6ce705d1152df6c84ebba2e21 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3-deactivate, RELEASE=squid-db0330b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-24T11:28:54.977 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:54 smithi062 podman[149251]: 2024-03-24 11:28:54.731701641 +0000 UTC m=+0.728148513 container died 05e9b6a05b76fb14d9616d5381d26c066522a8a6ce705d1152df6c84ebba2e21 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-03-24T11:28:55.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:55 smithi062 ceph-mon[90553]: Health check failed: Degraded data redundancy: 22/630 objects degraded (3.492%), 5 pgs degraded (PG_DEGRADED) 2024-03-24T11:28:55.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:55 smithi062 ceph-mon[98643]: Health check failed: Degraded data redundancy: 22/630 objects degraded (3.492%), 5 pgs degraded (PG_DEGRADED) 2024-03-24T11:28:55.478 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:55 smithi062 podman[149369]: 2024-03-24 11:28:55.347276558 +0000 UTC m=+0.607644926 container remove 05e9b6a05b76fb14d9616d5381d26c066522a8a6ce705d1152df6c84ebba2e21 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, RELEASE=squid-db0330b) 2024-03-24T11:28:55.478 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:55 smithi062 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.3.service: Deactivated successfully. 2024-03-24T11:28:55.478 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:55 smithi062 systemd[1]: Stopped Ceph osd.3 for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:28:55.478 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:55 smithi062 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.3.service: Consumed 1.986s CPU time. 2024-03-24T11:28:55.527 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:55 smithi139 ceph-mon[91894]: Health check failed: Degraded data redundancy: 22/630 objects degraded (3.492%), 5 pgs degraded (PG_DEGRADED) 2024-03-24T11:28:55.748 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:55 smithi062 systemd[1]: Starting Ceph osd.3 for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:28:56.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:56 smithi139 ceph-mon[91894]: pgmap v199: 161 pgs: 5 active+undersized+degraded, 7 peering, 21 stale+active+clean, 9 active+undersized, 119 active+clean; 585 KiB data, 191 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 22/630 objects degraded (3.492%) 2024-03-24T11:28:56.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:56.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:28:56.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:56 smithi062 ceph-mon[90553]: pgmap v199: 161 pgs: 5 active+undersized+degraded, 7 peering, 21 stale+active+clean, 9 active+undersized, 119 active+clean; 585 KiB data, 191 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 22/630 objects degraded (3.492%) 2024-03-24T11:28:56.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:56.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:28:56.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:56 smithi062 ceph-mon[98643]: pgmap v199: 161 pgs: 5 active+undersized+degraded, 7 peering, 21 stale+active+clean, 9 active+undersized, 119 active+clean; 585 KiB data, 191 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 22/630 objects degraded (3.492%) 2024-03-24T11:28:56.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:56.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:28:56.774 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:56 smithi062 podman[149554]: 2024-03-24 11:28:56.55282576 +0000 UTC m=+0.020983602 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:28:56.775 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:56 smithi062 podman[149554]: 2024-03-24 11:28:56.675571811 +0000 UTC m=+0.143729633 container create 9aac03a04f1b20567e33369fadc1d46c3c9e346c819c0a4811ab4ad124beb1a2 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3-activate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, RELEASE=squid-db0330b, CEPH_POINT_RELEASE=) 2024-03-24T11:28:57.028 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:56 smithi062 podman[149554]: 2024-03-24 11:28:56.878191696 +0000 UTC m=+0.346349532 container init 9aac03a04f1b20567e33369fadc1d46c3c9e346c819c0a4811ab4ad124beb1a2 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3-activate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-db0330b) 2024-03-24T11:28:57.029 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:56 smithi062 podman[149554]: 2024-03-24 11:28:56.881292642 +0000 UTC m=+0.349450475 container start 9aac03a04f1b20567e33369fadc1d46c3c9e346c819c0a4811ab4ad124beb1a2 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3-activate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, RELEASE=squid-db0330b) 2024-03-24T11:28:57.313 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:57 smithi062 podman[149554]: 2024-03-24 11:28:57.02841662 +0000 UTC m=+0.496574462 container attach 9aac03a04f1b20567e33369fadc1d46c3c9e346c819c0a4811ab4ad124beb1a2 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3-activate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS) 2024-03-24T11:28:57.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:57.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:57.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:57.686 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:57.687 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:57.687 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:57.687 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:57.687 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:57.687 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:28:57.687 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:57 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3-activate[149586]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-24T11:28:57.687 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:57 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3-activate[149586]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_1 2024-03-24T11:28:57.688 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:57 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3-activate[149586]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_1 2024-03-24T11:28:57.688 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:57 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3-activate[149586]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-0 2024-03-24T11:28:57.688 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:57 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3-activate[149586]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-3/block 2024-03-24T11:28:57.688 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:57 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3-activate[149586]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-24T11:28:57.688 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:57 smithi062 bash[149554]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-24T11:28:57.688 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:57 smithi062 bash[149554]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_1 2024-03-24T11:28:57.688 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:57 smithi062 bash[149554]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_1 2024-03-24T11:28:57.688 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:57 smithi062 bash[149554]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-0 2024-03-24T11:28:57.688 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:57 smithi062 bash[149554]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-3/block 2024-03-24T11:28:57.688 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:57 smithi062 bash[149554]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-24T11:28:57.688 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:57 smithi062 bash[149554]: --> ceph-volume raw activate successful for osd ID: 3 2024-03-24T11:28:57.688 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:57 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3-activate[149586]: --> ceph-volume raw activate successful for osd ID: 3 2024-03-24T11:28:57.688 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:57 smithi062 podman[149554]: 2024-03-24 11:28:57.686972301 +0000 UTC m=+1.155130115 container died 9aac03a04f1b20567e33369fadc1d46c3c9e346c819c0a4811ab4ad124beb1a2 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3-activate, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-24T11:28:58.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:28:58 smithi139 ceph-mon[91894]: pgmap v200: 161 pgs: 18 active+undersized+degraded, 7 peering, 6 stale+active+clean, 35 active+undersized, 95 active+clean; 585 KiB data, 191 MiB used, 715 GiB / 715 GiB avail; 60/630 objects degraded (9.524%) 2024-03-24T11:28:58.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:28:58 smithi062 ceph-mon[90553]: pgmap v200: 161 pgs: 18 active+undersized+degraded, 7 peering, 6 stale+active+clean, 35 active+undersized, 95 active+clean; 585 KiB data, 191 MiB used, 715 GiB / 715 GiB avail; 60/630 objects degraded (9.524%) 2024-03-24T11:28:58.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:28:58 smithi062 ceph-mon[98643]: pgmap v200: 161 pgs: 18 active+undersized+degraded, 7 peering, 6 stale+active+clean, 35 active+undersized, 95 active+clean; 585 KiB data, 191 MiB used, 715 GiB / 715 GiB avail; 60/630 objects degraded (9.524%) 2024-03-24T11:28:58.728 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:58 smithi062 podman[149743]: 2024-03-24 11:28:58.393383467 +0000 UTC m=+1.030491459 container remove 9aac03a04f1b20567e33369fadc1d46c3c9e346c819c0a4811ab4ad124beb1a2 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3-activate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True) 2024-03-24T11:28:59.217 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:58 smithi062 podman[149853]: 2024-03-24 11:28:58.76081468 +0000 UTC m=+0.023600620 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:28:59.217 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:58 smithi062 podman[149853]: 2024-03-24 11:28:58.896174462 +0000 UTC m=+0.158960395 container create 7501d2e4c718608a078f33a6d3f3096aa4bdb785d93f758443445ba420af4f77 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307) 2024-03-24T11:29:00.123 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:59 smithi062 podman[149853]: 2024-03-24 11:28:59.744096193 +0000 UTC m=+1.006882136 container init 7501d2e4c718608a078f33a6d3f3096aa4bdb785d93f758443445ba420af4f77 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3, org.label-schema.license=GPLv2, RELEASE=squid-db0330b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-24T11:29:00.123 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:59 smithi062 podman[149853]: 2024-03-24 11:28:59.74728622 +0000 UTC m=+1.010072154 container start 7501d2e4c718608a078f33a6d3f3096aa4bdb785d93f758443445ba420af4f77 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-db0330b, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS) 2024-03-24T11:29:00.123 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:59 smithi062 ceph-osd[149948]: -- 172.21.15.62:0/4220399228 <== mon.0 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5564ca42b1e0 con 0x5564ca3fd400 2024-03-24T11:29:00.123 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:59 smithi062 bash[149853]: 7501d2e4c718608a078f33a6d3f3096aa4bdb785d93f758443445ba420af4f77 2024-03-24T11:29:00.123 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:28:59 smithi062 systemd[1]: Started Ceph osd.3 for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:29:00.381 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3[149887]: 2024-03-24T11:29:00.290+0000 7fccc8b087c0 -1 Falling back to public interface 2024-03-24T11:29:00.961 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:29:00] "GET /metrics HTTP/1.1" 200 38519 "" "Prometheus/2.33.4" 2024-03-24T11:29:00.961 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[90553]: pgmap v201: 161 pgs: 23 active+undersized+degraded, 43 active+undersized, 95 active+clean; 585 KiB data, 191 MiB used, 715 GiB / 715 GiB avail; 80/630 objects degraded (12.698%) 2024-03-24T11:29:00.962 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:00.962 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:00.962 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:00.962 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:00.962 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:00.962 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:00.962 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:00.962 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:00.962 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:00.962 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:00.962 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:00.962 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:00.963 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:00.963 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:00.963 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:00.963 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:00.963 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:00.963 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:00.963 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:00.963 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:00.963 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:00.963 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:00.963 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:00.963 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:00.963 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:00.963 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:00.964 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:00.964 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:00.964 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[98643]: pgmap v201: 161 pgs: 23 active+undersized+degraded, 43 active+undersized, 95 active+clean; 585 KiB data, 191 MiB used, 715 GiB / 715 GiB avail; 80/630 objects degraded (12.698%) 2024-03-24T11:29:00.964 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:00.964 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:00.964 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:00.964 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:00.965 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:00.965 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:00.965 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:00.965 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:00.965 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:00.965 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:00.965 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:00.965 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:00.965 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:00.965 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:00.965 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:00.965 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:00.965 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:00.966 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:00.966 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:00.966 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:00.966 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:00.966 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:00.966 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:00.966 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:00.966 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:00.966 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:00.966 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:00.966 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:00.966 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:00.966 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:00.967 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:00.967 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:00.967 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:00.967 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:00.967 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:00.967 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:00.967 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:00.967 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:00.967 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:00.967 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:00.967 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:00.967 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:00.967 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:00.967 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:00.967 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:00.968 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:00.968 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:00.968 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:00.968 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:00.968 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:00.968 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:00.968 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:00.968 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:00.968 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:00.968 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:00.968 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:00.968 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:00.969 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:00.969 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:00.969 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:00.969 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:00.969 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:00.969 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:00.969 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:00.969 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:00.969 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:00.969 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:00.969 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:00.969 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:00.970 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:01.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:00 smithi139 ceph-mon[91894]: pgmap v201: 161 pgs: 23 active+undersized+degraded, 43 active+undersized, 95 active+clean; 585 KiB data, 191 MiB used, 715 GiB / 715 GiB avail; 80/630 objects degraded (12.698%) 2024-03-24T11:29:01.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:01.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:01.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:01.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:01.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:01.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:00 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:01.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:01.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:01.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:01.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:01.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:01.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:01.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:01.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:01.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:01.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:01.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:01.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:01.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:01.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:01.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:01.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:01.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:01.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:01.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:01.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:01.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:01.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:01.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:01.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:01.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:01.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:01.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:01.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:01.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:01.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:01.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:01.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:01.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:01.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:01.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:01.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:01.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:01.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:01.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:01.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:01.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:01.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:01.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:01.913 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:01.913 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:01.913 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:01.913 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:01.913 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:01.913 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:01.913 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:01.913 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:01.913 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:01.913 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:01.914 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:01.914 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:01.914 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:01.914 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:01.914 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:01.914 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:01.914 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:01.914 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:01.914 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:01.914 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:01.914 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:01.914 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:01.914 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:01.914 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:01.915 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:01.915 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:01.915 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:01.915 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:01.915 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:01.915 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:01.915 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:01.916 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:01.916 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:01.916 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:01.916 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:01.916 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:01.916 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:01.916 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:01.916 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:01.916 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:01.916 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:01.916 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:01.916 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:01.916 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:01.916 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:01.916 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:01.917 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:01.917 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:01.917 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:01.917 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:01.917 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:01.917 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:01.917 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:01.917 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:01.917 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:01.917 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:01.917 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:01.917 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:01.917 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:01.917 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:01.917 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:01.918 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:01.918 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:01.918 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:01.918 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:01.918 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:01.918 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:01.918 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:01.918 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:01.918 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:01.918 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:01.918 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:01.919 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:01.919 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:01.919 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:01.919 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:01.919 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:01.919 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:01.919 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:01.919 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:01.919 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:01.919 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:01.919 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:01.919 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:01.920 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:01.920 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:01.920 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:01.920 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:01.920 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:01.920 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:01.920 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:01.920 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:01.920 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:01.920 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:01.920 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:01.920 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:01.920 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:01.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:02.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:29:02.036+0000 7f4e4aad7700 -1 mgr.server reply reply (16) Device or resource busy unsafe to stop osd(s) at this time (20 PGs are or would become offline) 2024-03-24T11:29:02.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:01 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:02.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:01 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:02.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:01 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:02.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:01 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:02.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:01 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:02.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:01 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:02.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:01 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:02.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:01 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:02.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:02.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:02.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:02.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:02.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:02.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:01 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:02.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:02.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:02.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:02.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:02.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:02.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:02.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:02.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:02.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:02.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:02.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:02.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:02.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:02.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:02.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:02.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:02.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:02.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:02.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:02.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:02.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:02.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:02.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:02.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:02.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:02.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:02.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:02.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:02.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:02.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:02.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:02.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:02.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:02.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:02.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:03.103 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:29:03 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3[149887]: 2024-03-24T11:29:03.025+0000 7fccc8b087c0 -1 osd.3 143 log_to_monitors true 2024-03-24T11:29:03.103 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:03.103 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[98643]: pgmap v202: 161 pgs: 23 active+undersized+degraded, 43 active+undersized, 95 active+clean; 585 KiB data, 191 MiB used, 715 GiB / 715 GiB avail; 80/630 objects degraded (12.698%) 2024-03-24T11:29:03.103 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:03.103 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:03.103 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:03.104 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:03.104 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:03.104 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:03.104 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:03.104 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:03.104 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:03.104 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:03.104 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:03.104 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:03.104 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:03.104 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:03.104 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:03.104 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[98643]: pgmap v203: 161 pgs: 23 active+undersized+degraded, 43 active+undersized, 95 active+clean; 585 KiB data, 191 MiB used, 715 GiB / 715 GiB avail; 80/630 objects degraded (12.698%) 2024-03-24T11:29:03.104 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:03.105 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:03.105 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:03.105 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:03.105 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-24T11:29:03.105 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[98643]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-24T11:29:03.105 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[98643]: Upgrade: unsafe to stop osd(s) at this time (20 PGs are or would become offline) 2024-03-24T11:29:03.105 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[98643]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-03-24T11:29:03.105 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:03.105 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[90553]: pgmap v202: 161 pgs: 23 active+undersized+degraded, 43 active+undersized, 95 active+clean; 585 KiB data, 191 MiB used, 715 GiB / 715 GiB avail; 80/630 objects degraded (12.698%) 2024-03-24T11:29:03.105 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:03.105 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:03.105 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:03.106 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:03.106 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:03.106 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:03.106 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:03.106 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:03.106 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:03.106 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:03.106 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:03.106 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:03.106 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:03.106 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:03.106 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:03.106 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[90553]: pgmap v203: 161 pgs: 23 active+undersized+degraded, 43 active+undersized, 95 active+clean; 585 KiB data, 191 MiB used, 715 GiB / 715 GiB avail; 80/630 objects degraded (12.698%) 2024-03-24T11:29:03.106 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:03.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:03.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:03.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:03.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-24T11:29:03.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[90553]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-24T11:29:03.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[90553]: Upgrade: unsafe to stop osd(s) at this time (20 PGs are or would become offline) 2024-03-24T11:29:03.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:02 smithi062 ceph-mon[90553]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-03-24T11:29:03.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:02 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:03.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:02 smithi139 ceph-mon[91894]: pgmap v202: 161 pgs: 23 active+undersized+degraded, 43 active+undersized, 95 active+clean; 585 KiB data, 191 MiB used, 715 GiB / 715 GiB avail; 80/630 objects degraded (12.698%) 2024-03-24T11:29:03.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:02 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:03.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:02 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:03.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:02 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:03.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:02 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:03.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:02 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:03.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:02 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:03.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:03.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:03.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:03.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:03.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:03.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:02 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:03.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:03.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:03.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:03.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:02 smithi139 ceph-mon[91894]: pgmap v203: 161 pgs: 23 active+undersized+degraded, 43 active+undersized, 95 active+clean; 585 KiB data, 191 MiB used, 715 GiB / 715 GiB avail; 80/630 objects degraded (12.698%) 2024-03-24T11:29:03.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:03.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:03.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:03.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:03.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-24T11:29:03.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:02 smithi139 ceph-mon[91894]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-24T11:29:03.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:02 smithi139 ceph-mon[91894]: Upgrade: unsafe to stop osd(s) at this time (20 PGs are or would become offline) 2024-03-24T11:29:03.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:02 smithi139 ceph-mon[91894]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-03-24T11:29:04.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:03 smithi062 ceph-mon[98643]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:29:04.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:03 smithi062 ceph-mon[98643]: from='osd.3 [v2:172.21.15.62:6826/328459685,v1:172.21.15.62:6827/328459685]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-24T11:29:04.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:03 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:29:04.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:03 smithi062 ceph-mon[90553]: from='osd.3 [v2:172.21.15.62:6826/328459685,v1:172.21.15.62:6827/328459685]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-24T11:29:04.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:03 smithi139 ceph-mon[91894]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:29:04.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:03 smithi139 ceph-mon[91894]: from='osd.3 [v2:172.21.15.62:6826/328459685,v1:172.21.15.62:6827/328459685]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-24T11:29:04.727 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:29:04 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3[149887]: 2024-03-24T11:29:04.376+0000 7fccb5f58700 -1 osd.3 143 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T11:29:05.039 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:04 smithi062 ceph-mon[90553]: from='osd.3 [v2:172.21.15.62:6826/328459685,v1:172.21.15.62:6827/328459685]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-24T11:29:05.039 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:04 smithi062 ceph-mon[90553]: osdmap e146: 8 total, 7 up, 8 in 2024-03-24T11:29:05.039 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:04 smithi062 ceph-mon[90553]: from='osd.3 [v2:172.21.15.62:6826/328459685,v1:172.21.15.62:6827/328459685]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:29:05.040 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:04 smithi062 ceph-mon[90553]: failed to encode map e146 with expected crc 2024-03-24T11:29:05.040 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:04 smithi062 ceph-mon[90553]: failed to encode map e146 with expected crc 2024-03-24T11:29:05.040 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:04 smithi062 ceph-mon[90553]: failed to encode map e146 with expected crc 2024-03-24T11:29:05.040 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:04 smithi062 ceph-mon[90553]: failed to encode map e146 with expected crc 2024-03-24T11:29:05.040 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:04 smithi062 ceph-mon[90553]: pgmap v205: 161 pgs: 23 active+undersized+degraded, 43 active+undersized, 95 active+clean; 585 KiB data, 192 MiB used, 715 GiB / 715 GiB avail; 80/630 objects degraded (12.698%) 2024-03-24T11:29:05.040 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:04 smithi062 ceph-mon[90553]: Health check update: Degraded data redundancy: 80/630 objects degraded (12.698%), 23 pgs degraded (PG_DEGRADED) 2024-03-24T11:29:05.040 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:04 smithi062 ceph-mon[90553]: from='osd.3 [v2:172.21.15.62:6826/328459685,v1:172.21.15.62:6827/328459685]' entity='osd.3' 2024-03-24T11:29:05.040 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:04 smithi062 ceph-mon[98643]: from='osd.3 [v2:172.21.15.62:6826/328459685,v1:172.21.15.62:6827/328459685]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-24T11:29:05.040 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:04 smithi062 ceph-mon[98643]: osdmap e146: 8 total, 7 up, 8 in 2024-03-24T11:29:05.040 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:04 smithi062 ceph-mon[98643]: from='osd.3 [v2:172.21.15.62:6826/328459685,v1:172.21.15.62:6827/328459685]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:29:05.040 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:04 smithi062 ceph-mon[98643]: failed to encode map e146 with expected crc 2024-03-24T11:29:05.040 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:04 smithi062 ceph-mon[98643]: failed to encode map e146 with expected crc 2024-03-24T11:29:05.041 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:04 smithi062 ceph-mon[98643]: failed to encode map e146 with expected crc 2024-03-24T11:29:05.041 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:04 smithi062 ceph-mon[98643]: failed to encode map e146 with expected crc 2024-03-24T11:29:05.041 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:04 smithi062 ceph-mon[98643]: pgmap v205: 161 pgs: 23 active+undersized+degraded, 43 active+undersized, 95 active+clean; 585 KiB data, 192 MiB used, 715 GiB / 715 GiB avail; 80/630 objects degraded (12.698%) 2024-03-24T11:29:05.041 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:04 smithi062 ceph-mon[98643]: Health check update: Degraded data redundancy: 80/630 objects degraded (12.698%), 23 pgs degraded (PG_DEGRADED) 2024-03-24T11:29:05.041 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:04 smithi062 ceph-mon[98643]: from='osd.3 [v2:172.21.15.62:6826/328459685,v1:172.21.15.62:6827/328459685]' entity='osd.3' 2024-03-24T11:29:05.259 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:04 smithi139 ceph-mon[91894]: from='osd.3 [v2:172.21.15.62:6826/328459685,v1:172.21.15.62:6827/328459685]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-24T11:29:05.260 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:04 smithi139 ceph-mon[91894]: osdmap e146: 8 total, 7 up, 8 in 2024-03-24T11:29:05.260 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:04 smithi139 ceph-mon[91894]: from='osd.3 [v2:172.21.15.62:6826/328459685,v1:172.21.15.62:6827/328459685]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-03-24T11:29:05.260 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:04 smithi139 ceph-mon[91894]: failed to encode map e146 with expected crc 2024-03-24T11:29:05.260 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:04 smithi139 ceph-mon[91894]: failed to encode map e146 with expected crc 2024-03-24T11:29:05.260 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:04 smithi139 ceph-mon[91894]: failed to encode map e146 with expected crc 2024-03-24T11:29:05.260 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:04 smithi139 ceph-mon[91894]: failed to encode map e146 with expected crc 2024-03-24T11:29:05.260 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:04 smithi139 ceph-mon[91894]: pgmap v205: 161 pgs: 23 active+undersized+degraded, 43 active+undersized, 95 active+clean; 585 KiB data, 192 MiB used, 715 GiB / 715 GiB avail; 80/630 objects degraded (12.698%) 2024-03-24T11:29:05.260 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:04 smithi139 ceph-mon[91894]: Health check update: Degraded data redundancy: 80/630 objects degraded (12.698%), 23 pgs degraded (PG_DEGRADED) 2024-03-24T11:29:05.260 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:04 smithi139 ceph-mon[91894]: from='osd.3 [v2:172.21.15.62:6826/328459685,v1:172.21.15.62:6827/328459685]' entity='osd.3' 2024-03-24T11:29:05.638 INFO:teuthology.orchestra.run.smithi062.stdout:true 2024-03-24T11:29:06.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:05 smithi062 ceph-mon[90553]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T11:29:06.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:05 smithi062 ceph-mon[90553]: osd.3 [v2:172.21.15.62:6826/328459685,v1:172.21.15.62:6827/328459685] boot 2024-03-24T11:29:06.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:05 smithi062 ceph-mon[90553]: osdmap e147: 8 total, 8 up, 8 in 2024-03-24T11:29:06.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:05 smithi062 ceph-mon[90553]: failed to encode map e147 with expected crc 2024-03-24T11:29:06.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:05 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T11:29:06.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:05 smithi062 ceph-mon[90553]: failed to encode map e147 with expected crc 2024-03-24T11:29:06.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:05 smithi062 ceph-mon[98643]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T11:29:06.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:05 smithi062 ceph-mon[98643]: osd.3 [v2:172.21.15.62:6826/328459685,v1:172.21.15.62:6827/328459685] boot 2024-03-24T11:29:06.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:05 smithi062 ceph-mon[98643]: osdmap e147: 8 total, 8 up, 8 in 2024-03-24T11:29:06.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:05 smithi062 ceph-mon[98643]: failed to encode map e147 with expected crc 2024-03-24T11:29:06.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:05 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T11:29:06.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:05 smithi062 ceph-mon[98643]: failed to encode map e147 with expected crc 2024-03-24T11:29:06.276 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:05 smithi139 ceph-mon[91894]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T11:29:06.276 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:05 smithi139 ceph-mon[91894]: osd.3 [v2:172.21.15.62:6826/328459685,v1:172.21.15.62:6827/328459685] boot 2024-03-24T11:29:06.276 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:05 smithi139 ceph-mon[91894]: osdmap e147: 8 total, 8 up, 8 in 2024-03-24T11:29:06.276 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:05 smithi139 ceph-mon[91894]: failed to encode map e147 with expected crc 2024-03-24T11:29:06.276 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:05 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T11:29:06.276 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:05 smithi139 ceph-mon[91894]: failed to encode map e147 with expected crc 2024-03-24T11:29:06.403 INFO:teuthology.orchestra.run.smithi062.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T11:29:06.403 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi062 smithi062 running 4s ago 18m - - 2024-03-24T11:29:06.403 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi139 smithi139 running 6s ago 18m - - 2024-03-24T11:29:06.403 INFO:teuthology.orchestra.run.smithi062.stdout:alertmanager.a smithi062 *:9093,9094 running (8m) 4s ago 13m 24.8M - 0.23.0 ba2b418f427c 818298d58140 2024-03-24T11:29:06.404 INFO:teuthology.orchestra.run.smithi062.stdout:grafana.a smithi139 *:3000 running (7m) 6s ago 12m 53.2M - 8.3.5 dad864ee21e9 b48167e7171e 2024-03-24T11:29:06.404 INFO:teuthology.orchestra.run.smithi062.stdout:iscsi.foo.smithi062.dggsvk smithi062 running (6m) 4s ago 11m 64.1M - 3.5 e1d6a67b021e 0a98806ec839 2024-03-24T11:29:06.404 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.x smithi139 *:8443,9283,8765 running (5m) 6s ago 17m 452M - 19.0.0-1667-gdb0330b1 4f325f8440e9 d94987b11387 2024-03-24T11:29:06.404 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.y smithi062 *:8443,9283,8765 running (7m) 4s ago 19m 527M - 19.0.0-1667-gdb0330b1 4f325f8440e9 ca1a49028d65 2024-03-24T11:29:06.404 INFO:teuthology.orchestra.run.smithi062.stdout:mon.a smithi062 running (5m) 4s ago 19m 65.4M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 f7afe6e28c88 2024-03-24T11:29:06.404 INFO:teuthology.orchestra.run.smithi062.stdout:mon.b smithi139 running (4m) 6s ago 18m 47.6M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 ca357beea4dd 2024-03-24T11:29:06.404 INFO:teuthology.orchestra.run.smithi062.stdout:mon.c smithi062 running (4m) 4s ago 18m 56.9M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 459e7449ec39 2024-03-24T11:29:06.404 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.a smithi062 *:9100 running (8m) 4s ago 13m 15.8M - 1.3.1 1dbe0e931976 79d5a02a999f 2024-03-24T11:29:06.404 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.b smithi139 *:9100 running (7m) 6s ago 13m 15.9M - 1.3.1 1dbe0e931976 8f9f4608e465 2024-03-24T11:29:06.404 INFO:teuthology.orchestra.run.smithi062.stdout:osd.0 smithi062 running (3m) 4s ago 17m 55.5M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 dfe4920b49c4 2024-03-24T11:29:06.404 INFO:teuthology.orchestra.run.smithi062.stdout:osd.1 smithi062 running (2m) 4s ago 17m 47.1M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 c443b3fe97c1 2024-03-24T11:29:06.404 INFO:teuthology.orchestra.run.smithi062.stdout:osd.2 smithi062 running (69s) 4s ago 16m 44.2M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 0eba40dc96b3 2024-03-24T11:29:06.404 INFO:teuthology.orchestra.run.smithi062.stdout:osd.3 smithi062 unknown 4s ago 16m - 4096M 2024-03-24T11:29:06.404 INFO:teuthology.orchestra.run.smithi062.stdout:osd.4 smithi139 running (15m) 6s ago 15m 69.0M 4096M 17.2.0 e1d6a67b021e 7568bfa9e74d 2024-03-24T11:29:06.404 INFO:teuthology.orchestra.run.smithi062.stdout:osd.5 smithi139 running (15m) 6s ago 15m 62.2M 4096M 17.2.0 e1d6a67b021e dd0abd599a14 2024-03-24T11:29:06.405 INFO:teuthology.orchestra.run.smithi062.stdout:osd.6 smithi139 running (15m) 6s ago 15m 61.2M 4096M 17.2.0 e1d6a67b021e a4b1289aca69 2024-03-24T11:29:06.405 INFO:teuthology.orchestra.run.smithi062.stdout:osd.7 smithi139 running (14m) 6s ago 14m 64.8M 4096M 17.2.0 e1d6a67b021e bc7fadbc1384 2024-03-24T11:29:06.405 INFO:teuthology.orchestra.run.smithi062.stdout:prometheus.a smithi139 *:9095 running (6m) 6s ago 14m 71.2M - 2.33.4 514e6a882f6e 2ed47f810d33 2024-03-24T11:29:06.405 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi062.sacpds smithi062 *:8000 running (12m) 4s ago 12m 74.6M - 17.2.0 e1d6a67b021e 89f4ac308e98 2024-03-24T11:29:06.405 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi139.yljesh smithi139 *:8000 running (12m) 6s ago 12m 73.7M - 17.2.0 e1d6a67b021e e189bdc12cae 2024-03-24T11:29:06.405 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi062.xvpxnk smithi062 *:80 running (12m) 4s ago 12m 74.4M - 17.2.0 e1d6a67b021e d1aa74adab44 2024-03-24T11:29:06.405 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi139.pbtrmp smithi139 *:80 running (12m) 6s ago 12m 77.4M - 17.2.0 e1d6a67b021e 7f8e336755d7 2024-03-24T11:29:06.846 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:29:06.846 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": { 2024-03-24T11:29:06.846 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 3 2024-03-24T11:29:06.846 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:29:06.846 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": { 2024-03-24T11:29:06.847 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-24T11:29:06.847 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:29:06.847 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": { 2024-03-24T11:29:06.847 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4, 2024-03-24T11:29:06.847 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 4 2024-03-24T11:29:06.847 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:29:06.847 INFO:teuthology.orchestra.run.smithi062.stdout: "rgw": { 2024-03-24T11:29:06.847 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4 2024-03-24T11:29:06.847 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:29:06.847 INFO:teuthology.orchestra.run.smithi062.stdout: "overall": { 2024-03-24T11:29:06.847 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 8, 2024-03-24T11:29:06.847 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 9 2024-03-24T11:29:06.847 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-24T11:29:06.848 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:29:07.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:06 smithi139 ceph-mon[91894]: failed to encode map e147 with expected crc 2024-03-24T11:29:07.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:06 smithi139 ceph-mon[91894]: failed to encode map e147 with expected crc 2024-03-24T11:29:07.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:06 smithi139 ceph-mon[91894]: failed to encode map e147 with expected crc 2024-03-24T11:29:07.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:06 smithi139 ceph-mon[91894]: failed to encode map e147 with expected crc 2024-03-24T11:29:07.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:06 smithi139 ceph-mon[91894]: failed to encode map e147 with expected crc 2024-03-24T11:29:07.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:06 smithi139 ceph-mon[91894]: failed to encode map e147 with expected crc 2024-03-24T11:29:07.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:06 smithi139 ceph-mon[91894]: from='client.44469 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:29:07.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:06 smithi139 ceph-mon[91894]: from='client.44475 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:29:07.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:06 smithi139 ceph-mon[91894]: pgmap v207: 161 pgs: 23 active+undersized+degraded, 43 active+undersized, 95 active+clean; 585 KiB data, 192 MiB used, 715 GiB / 715 GiB avail; 742 B/s rd, 0 op/s; 80/630 objects degraded (12.698%) 2024-03-24T11:29:07.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:06 smithi139 ceph-mon[91894]: failed to encode map e148 with expected crc 2024-03-24T11:29:07.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:06 smithi139 ceph-mon[91894]: osdmap e148: 8 total, 8 up, 8 in 2024-03-24T11:29:07.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:06 smithi139 ceph-mon[91894]: failed to encode map e148 with expected crc 2024-03-24T11:29:07.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:06 smithi139 ceph-mon[91894]: from='client.? 172.21.15.62:0/3865964226' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:07.194 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:29:07.195 INFO:teuthology.orchestra.run.smithi062.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69", 2024-03-24T11:29:07.195 INFO:teuthology.orchestra.run.smithi062.stdout: "in_progress": true, 2024-03-24T11:29:07.195 INFO:teuthology.orchestra.run.smithi062.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T11:29:07.195 INFO:teuthology.orchestra.run.smithi062.stdout: "services_complete": [ 2024-03-24T11:29:07.195 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr", 2024-03-24T11:29:07.195 INFO:teuthology.orchestra.run.smithi062.stdout: "mon" 2024-03-24T11:29:07.195 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-03-24T11:29:07.195 INFO:teuthology.orchestra.run.smithi062.stdout: "progress": "8/23 daemons upgraded", 2024-03-24T11:29:07.195 INFO:teuthology.orchestra.run.smithi062.stdout: "message": "Currently upgrading osd daemons", 2024-03-24T11:29:07.195 INFO:teuthology.orchestra.run.smithi062.stdout: "is_paused": false 2024-03-24T11:29:07.195 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:29:07.196 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:06 smithi062 ceph-mon[98643]: failed to encode map e147 with expected crc 2024-03-24T11:29:07.196 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:06 smithi062 ceph-mon[98643]: failed to encode map e147 with expected crc 2024-03-24T11:29:07.196 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:06 smithi062 ceph-mon[98643]: failed to encode map e147 with expected crc 2024-03-24T11:29:07.196 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:06 smithi062 ceph-mon[98643]: failed to encode map e147 with expected crc 2024-03-24T11:29:07.196 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:06 smithi062 ceph-mon[98643]: failed to encode map e147 with expected crc 2024-03-24T11:29:07.196 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:06 smithi062 ceph-mon[98643]: failed to encode map e147 with expected crc 2024-03-24T11:29:07.196 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:06 smithi062 ceph-mon[98643]: from='client.44469 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:29:07.196 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:06 smithi062 ceph-mon[98643]: from='client.44475 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:29:07.196 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:06 smithi062 ceph-mon[98643]: pgmap v207: 161 pgs: 23 active+undersized+degraded, 43 active+undersized, 95 active+clean; 585 KiB data, 192 MiB used, 715 GiB / 715 GiB avail; 742 B/s rd, 0 op/s; 80/630 objects degraded (12.698%) 2024-03-24T11:29:07.196 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:06 smithi062 ceph-mon[98643]: failed to encode map e148 with expected crc 2024-03-24T11:29:07.196 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:06 smithi062 ceph-mon[98643]: osdmap e148: 8 total, 8 up, 8 in 2024-03-24T11:29:07.197 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:06 smithi062 ceph-mon[98643]: failed to encode map e148 with expected crc 2024-03-24T11:29:07.197 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:06 smithi062 ceph-mon[98643]: from='client.? 172.21.15.62:0/3865964226' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:07.197 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:06 smithi062 ceph-mon[90553]: failed to encode map e147 with expected crc 2024-03-24T11:29:07.197 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:06 smithi062 ceph-mon[90553]: failed to encode map e147 with expected crc 2024-03-24T11:29:07.197 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:06 smithi062 ceph-mon[90553]: failed to encode map e147 with expected crc 2024-03-24T11:29:07.197 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:06 smithi062 ceph-mon[90553]: failed to encode map e147 with expected crc 2024-03-24T11:29:07.197 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:06 smithi062 ceph-mon[90553]: failed to encode map e147 with expected crc 2024-03-24T11:29:07.197 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:06 smithi062 ceph-mon[90553]: failed to encode map e147 with expected crc 2024-03-24T11:29:07.197 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:06 smithi062 ceph-mon[90553]: from='client.44469 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:29:07.197 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:06 smithi062 ceph-mon[90553]: from='client.44475 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:29:07.197 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:06 smithi062 ceph-mon[90553]: pgmap v207: 161 pgs: 23 active+undersized+degraded, 43 active+undersized, 95 active+clean; 585 KiB data, 192 MiB used, 715 GiB / 715 GiB avail; 742 B/s rd, 0 op/s; 80/630 objects degraded (12.698%) 2024-03-24T11:29:07.197 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:06 smithi062 ceph-mon[90553]: failed to encode map e148 with expected crc 2024-03-24T11:29:07.197 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:06 smithi062 ceph-mon[90553]: osdmap e148: 8 total, 8 up, 8 in 2024-03-24T11:29:07.198 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:06 smithi062 ceph-mon[90553]: failed to encode map e148 with expected crc 2024-03-24T11:29:07.198 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:06 smithi062 ceph-mon[90553]: from='client.? 172.21.15.62:0/3865964226' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:07.654 INFO:teuthology.orchestra.run.smithi062.stdout:HEALTH_WARN 1 failed cephadm daemon(s); Degraded data redundancy: 80/630 objects degraded (12.698%), 23 pgs degraded 2024-03-24T11:29:07.655 INFO:teuthology.orchestra.run.smithi062.stdout:[WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-03-24T11:29:07.655 INFO:teuthology.orchestra.run.smithi062.stdout: daemon osd.3 on smithi062 is in unknown state 2024-03-24T11:29:07.655 INFO:teuthology.orchestra.run.smithi062.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 80/630 objects degraded (12.698%), 23 pgs degraded 2024-03-24T11:29:07.655 INFO:teuthology.orchestra.run.smithi062.stdout: pg 2.9 is active+undersized+degraded, acting [1,7] 2024-03-24T11:29:07.655 INFO:teuthology.orchestra.run.smithi062.stdout: pg 2.a is active+undersized+degraded, acting [1,7] 2024-03-24T11:29:07.655 INFO:teuthology.orchestra.run.smithi062.stdout: pg 2.e is active+undersized+degraded, acting [2,7] 2024-03-24T11:29:07.655 INFO:teuthology.orchestra.run.smithi062.stdout: pg 2.12 is active+undersized+degraded, acting [5,7] 2024-03-24T11:29:07.655 INFO:teuthology.orchestra.run.smithi062.stdout: pg 3.1 is active+undersized+degraded, acting [0,4] 2024-03-24T11:29:07.655 INFO:teuthology.orchestra.run.smithi062.stdout: pg 3.2 is active+undersized+degraded, acting [5,6] 2024-03-24T11:29:07.655 INFO:teuthology.orchestra.run.smithi062.stdout: pg 3.5 is active+undersized+degraded, acting [5,2] 2024-03-24T11:29:07.655 INFO:teuthology.orchestra.run.smithi062.stdout: pg 3.7 is active+undersized+degraded, acting [7,0] 2024-03-24T11:29:07.655 INFO:teuthology.orchestra.run.smithi062.stdout: pg 3.8 is active+undersized+degraded, acting [1,7] 2024-03-24T11:29:07.655 INFO:teuthology.orchestra.run.smithi062.stdout: pg 3.b is active+undersized+degraded, acting [0,4] 2024-03-24T11:29:07.655 INFO:teuthology.orchestra.run.smithi062.stdout: pg 3.c is active+undersized+degraded, acting [5,6] 2024-03-24T11:29:07.656 INFO:teuthology.orchestra.run.smithi062.stdout: pg 3.12 is active+undersized+degraded, acting [0,7] 2024-03-24T11:29:07.656 INFO:teuthology.orchestra.run.smithi062.stdout: pg 3.15 is active+undersized+degraded, acting [7,4] 2024-03-24T11:29:07.656 INFO:teuthology.orchestra.run.smithi062.stdout: pg 3.17 is active+undersized+degraded, acting [0,5] 2024-03-24T11:29:07.656 INFO:teuthology.orchestra.run.smithi062.stdout: pg 3.18 is active+undersized+degraded, acting [0,1] 2024-03-24T11:29:07.656 INFO:teuthology.orchestra.run.smithi062.stdout: pg 3.19 is active+undersized+degraded, acting [1,4] 2024-03-24T11:29:07.656 INFO:teuthology.orchestra.run.smithi062.stdout: pg 3.1e is active+undersized+degraded, acting [6,2] 2024-03-24T11:29:07.656 INFO:teuthology.orchestra.run.smithi062.stdout: pg 4.0 is active+undersized+degraded, acting [7,0] 2024-03-24T11:29:07.656 INFO:teuthology.orchestra.run.smithi062.stdout: pg 4.9 is active+undersized+degraded, acting [4,1] 2024-03-24T11:29:07.656 INFO:teuthology.orchestra.run.smithi062.stdout: pg 4.14 is active+undersized+degraded, acting [1,7] 2024-03-24T11:29:07.656 INFO:teuthology.orchestra.run.smithi062.stdout: pg 4.15 is active+undersized+degraded, acting [5,7] 2024-03-24T11:29:07.656 INFO:teuthology.orchestra.run.smithi062.stdout: pg 5.1c is active+undersized+degraded, acting [4,2] 2024-03-24T11:29:07.656 INFO:teuthology.orchestra.run.smithi062.stdout: pg 6.c is active+undersized+degraded, acting [6,5] 2024-03-24T11:29:08.175 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:07 smithi062 ceph-mon[90553]: failed to encode map e148 with expected crc 2024-03-24T11:29:08.175 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:07 smithi062 ceph-mon[90553]: from='client.64450 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:29:08.175 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:07 smithi062 ceph-mon[90553]: failed to encode map e148 with expected crc 2024-03-24T11:29:08.175 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:07 smithi062 ceph-mon[90553]: failed to encode map e148 with expected crc 2024-03-24T11:29:08.175 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:07 smithi062 ceph-mon[90553]: failed to encode map e148 with expected crc 2024-03-24T11:29:08.175 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:07 smithi062 ceph-mon[90553]: failed to encode map e148 with expected crc 2024-03-24T11:29:08.175 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:07 smithi062 ceph-mon[90553]: failed to encode map e148 with expected crc 2024-03-24T11:29:08.175 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:07 smithi062 ceph-mon[90553]: failed to encode map e148 with expected crc 2024-03-24T11:29:08.175 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:07 smithi062 ceph-mon[90553]: from='client.? 172.21.15.62:0/1577741480' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:29:08.176 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:07 smithi062 ceph-mon[98643]: failed to encode map e148 with expected crc 2024-03-24T11:29:08.176 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:07 smithi062 ceph-mon[98643]: from='client.64450 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:29:08.176 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:07 smithi062 ceph-mon[98643]: failed to encode map e148 with expected crc 2024-03-24T11:29:08.176 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:07 smithi062 ceph-mon[98643]: failed to encode map e148 with expected crc 2024-03-24T11:29:08.176 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:07 smithi062 ceph-mon[98643]: failed to encode map e148 with expected crc 2024-03-24T11:29:08.176 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:07 smithi062 ceph-mon[98643]: failed to encode map e148 with expected crc 2024-03-24T11:29:08.176 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:07 smithi062 ceph-mon[98643]: failed to encode map e148 with expected crc 2024-03-24T11:29:08.176 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:07 smithi062 ceph-mon[98643]: failed to encode map e148 with expected crc 2024-03-24T11:29:08.176 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:07 smithi062 ceph-mon[98643]: from='client.? 172.21.15.62:0/1577741480' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:29:08.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:07 smithi139 ceph-mon[91894]: failed to encode map e148 with expected crc 2024-03-24T11:29:08.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:07 smithi139 ceph-mon[91894]: from='client.64450 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:29:08.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:07 smithi139 ceph-mon[91894]: failed to encode map e148 with expected crc 2024-03-24T11:29:08.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:07 smithi139 ceph-mon[91894]: failed to encode map e148 with expected crc 2024-03-24T11:29:08.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:07 smithi139 ceph-mon[91894]: failed to encode map e148 with expected crc 2024-03-24T11:29:08.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:07 smithi139 ceph-mon[91894]: failed to encode map e148 with expected crc 2024-03-24T11:29:08.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:07 smithi139 ceph-mon[91894]: failed to encode map e148 with expected crc 2024-03-24T11:29:08.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:07 smithi139 ceph-mon[91894]: failed to encode map e148 with expected crc 2024-03-24T11:29:08.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:07 smithi139 ceph-mon[91894]: from='client.? 172.21.15.62:0/1577741480' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:29:08.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:08 smithi062 ceph-mon[90553]: from='client.44493 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:29:08.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:08 smithi062 ceph-mon[90553]: pgmap v209: 161 pgs: 15 active+undersized+degraded, 27 active+undersized, 119 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 59/630 objects degraded (9.365%) 2024-03-24T11:29:08.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:08 smithi062 ceph-mon[98643]: from='client.44493 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:29:08.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:08 smithi062 ceph-mon[98643]: pgmap v209: 161 pgs: 15 active+undersized+degraded, 27 active+undersized, 119 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 59/630 objects degraded (9.365%) 2024-03-24T11:29:09.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:08 smithi139 ceph-mon[91894]: from='client.44493 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:29:09.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:08 smithi139 ceph-mon[91894]: pgmap v209: 161 pgs: 15 active+undersized+degraded, 27 active+undersized, 119 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 59/630 objects degraded (9.365%) 2024-03-24T11:29:10.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:10.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:10.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:10 smithi139 ceph-mon[91894]: pgmap v210: 161 pgs: 161 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:29:10.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:10.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:10 smithi139 ceph-mon[91894]: Health check update: Degraded data redundancy: 59/630 objects degraded (9.365%), 15 pgs degraded (PG_DEGRADED) 2024-03-24T11:29:10.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:10.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:10.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:10.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:10.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:10.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:10 smithi062 ceph-mon[90553]: pgmap v210: 161 pgs: 161 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:29:10.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:10.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:10 smithi062 ceph-mon[90553]: Health check update: Degraded data redundancy: 59/630 objects degraded (9.365%), 15 pgs degraded (PG_DEGRADED) 2024-03-24T11:29:10.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:10.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:10.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:10.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:10.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:10.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:10 smithi062 ceph-mon[98643]: pgmap v210: 161 pgs: 161 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:29:10.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:10.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:10 smithi062 ceph-mon[98643]: Health check update: Degraded data redundancy: 59/630 objects degraded (9.365%), 15 pgs degraded (PG_DEGRADED) 2024-03-24T11:29:10.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:10.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:10.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:11.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:29:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:29:10] "GET /metrics HTTP/1.1" 200 38519 "" "Prometheus/2.33.4" 2024-03-24T11:29:11.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:11 smithi139 ceph-mon[91894]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-03-24T11:29:11.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:11 smithi139 ceph-mon[91894]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 59/630 objects degraded (9.365%), 15 pgs degraded) 2024-03-24T11:29:11.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:11 smithi139 ceph-mon[91894]: Cluster is now healthy 2024-03-24T11:29:11.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:11.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:29:11.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:11 smithi062 ceph-mon[90553]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-03-24T11:29:11.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:11 smithi062 ceph-mon[90553]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 59/630 objects degraded (9.365%), 15 pgs degraded) 2024-03-24T11:29:11.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:11 smithi062 ceph-mon[90553]: Cluster is now healthy 2024-03-24T11:29:11.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:11.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:29:11.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:11 smithi062 ceph-mon[98643]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-03-24T11:29:11.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:11 smithi062 ceph-mon[98643]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 59/630 objects degraded (9.365%), 15 pgs degraded) 2024-03-24T11:29:11.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:11 smithi062 ceph-mon[98643]: Cluster is now healthy 2024-03-24T11:29:11.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:11.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:29:12.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:12 smithi139 ceph-mon[91894]: pgmap v211: 161 pgs: 161 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:29:12.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:12 smithi062 ceph-mon[90553]: pgmap v211: 161 pgs: 161 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:29:12.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:12 smithi062 ceph-mon[98643]: pgmap v211: 161 pgs: 161 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:29:13.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:13 smithi139 ceph-mon[91894]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:29:13.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:13 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:29:13.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:13 smithi062 ceph-mon[98643]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:29:14.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:14 smithi139 ceph-mon[91894]: pgmap v212: 161 pgs: 161 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:29:14.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:14 smithi062 ceph-mon[98643]: pgmap v212: 161 pgs: 161 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:29:14.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:14 smithi062 ceph-mon[90553]: pgmap v212: 161 pgs: 161 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:29:16.757 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:16 smithi139 ceph-mon[91894]: pgmap v213: 161 pgs: 161 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:29:16.757 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:16.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:16 smithi062 ceph-mon[90553]: pgmap v213: 161 pgs: 161 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:29:16.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:16.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:16 smithi062 ceph-mon[98643]: pgmap v213: 161 pgs: 161 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:29:16.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:17.580 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-24T11:29:17.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-24T11:29:17.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-24T11:29:18.711 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:18 smithi139 ceph-mon[91894]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-24T11:29:18.711 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:18 smithi139 ceph-mon[91894]: Upgrade: osd.4 is safe to restart 2024-03-24T11:29:18.711 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:18 smithi139 ceph-mon[91894]: pgmap v214: 161 pgs: 161 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:29:18.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:18 smithi062 ceph-mon[98643]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-24T11:29:18.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:18 smithi062 ceph-mon[98643]: Upgrade: osd.4 is safe to restart 2024-03-24T11:29:18.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:18 smithi062 ceph-mon[98643]: pgmap v214: 161 pgs: 161 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:29:18.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:18 smithi062 ceph-mon[90553]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-24T11:29:18.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:18 smithi062 ceph-mon[90553]: Upgrade: osd.4 is safe to restart 2024-03-24T11:29:18.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:18 smithi062 ceph-mon[90553]: pgmap v214: 161 pgs: 161 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:29:20.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:20 smithi139 ceph-mon[91894]: Upgrade: Updating osd.4 2024-03-24T11:29:20.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:20 smithi139 ceph-mon[91894]: pgmap v215: 161 pgs: 161 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:29:20.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:20.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-24T11:29:20.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:20.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:20 smithi139 ceph-mon[91894]: Deploying daemon osd.4 on smithi139 2024-03-24T11:29:20.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:20.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:20.573 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:20.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:20 smithi062 ceph-mon[98643]: Upgrade: Updating osd.4 2024-03-24T11:29:20.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:20 smithi062 ceph-mon[98643]: pgmap v215: 161 pgs: 161 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:29:20.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:20.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-24T11:29:20.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:20.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:20 smithi062 ceph-mon[98643]: Deploying daemon osd.4 on smithi139 2024-03-24T11:29:20.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:20.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:20.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:20.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:20 smithi062 ceph-mon[90553]: Upgrade: Updating osd.4 2024-03-24T11:29:20.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:20 smithi062 ceph-mon[90553]: pgmap v215: 161 pgs: 161 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:29:20.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:20.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-24T11:29:20.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:20.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:20 smithi062 ceph-mon[90553]: Deploying daemon osd.4 on smithi139 2024-03-24T11:29:20.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:20.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:20.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:21.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:29:20 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:29:20] "GET /metrics HTTP/1.1" 200 38515 "" "Prometheus/2.33.4" 2024-03-24T11:29:22.616 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:22 smithi062 ceph-mon[90553]: pgmap v216: 161 pgs: 161 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail; 842 B/s rd, 0 op/s 2024-03-24T11:29:22.617 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:22 smithi062 ceph-mon[98643]: pgmap v216: 161 pgs: 161 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail; 842 B/s rd, 0 op/s 2024-03-24T11:29:22.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:22 smithi139 ceph-mon[91894]: pgmap v216: 161 pgs: 161 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail; 842 B/s rd, 0 op/s 2024-03-24T11:29:22.858 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:22 smithi139 systemd[1]: Stopping Ceph osd.4 for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:29:23.358 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:22 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4[30328]: 2024-03-24T11:29:22.955+0000 7fdc3a583700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T11:29:23.358 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:22 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4[30328]: 2024-03-24T11:29:22.955+0000 7fdc3a583700 -1 osd.4 148 *** Got signal Terminated *** 2024-03-24T11:29:23.358 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:22 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4[30328]: 2024-03-24T11:29:22.955+0000 7fdc3a583700 -1 osd.4 148 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T11:29:23.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:23 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:29:23.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:23 smithi062 ceph-mon[90553]: osd.4 marked itself down and dead 2024-03-24T11:29:23.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:23 smithi062 ceph-mon[98643]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:29:23.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:23 smithi062 ceph-mon[98643]: osd.4 marked itself down and dead 2024-03-24T11:29:23.814 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:23 smithi139 ceph-mon[91894]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:29:23.814 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:23 smithi139 ceph-mon[91894]: osd.4 marked itself down and dead 2024-03-24T11:29:24.106 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:23 smithi139 podman[120132]: 2024-03-24 11:29:23.813885956 +0000 UTC m=+1.011121655 container died 7568bfa9e74d579ca629d94881a3ee86e263059f2b4b36f72314cf1f8c2ef7ea (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, io.buildah.version=1.19.8, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, CEPH_POINT_RELEASE=-17.2.0, maintainer=Guillaume Abrioux , architecture=x86_64, distribution-scope=public, io.k8s.display-name=CentOS Stream 8, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, name=centos-stream, io.openshift.tags=base centos centos-stream, version=8, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, GIT_BRANCH=HEAD, release=754, GIT_CLEAN=True, vcs-type=git, io.openshift.expose-services=, com.redhat.component=centos-stream-container, GIT_REPO=https://github.com/ceph/ceph-container.git, build-date=2022-05-03T08:36:31.336870, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, RELEASE=HEAD, ceph=True, vendor=Red Hat, Inc.) 2024-03-24T11:29:24.358 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:24 smithi139 podman[120132]: 2024-03-24 11:29:24.106180557 +0000 UTC m=+1.303416263 container cleanup 7568bfa9e74d579ca629d94881a3ee86e263059f2b4b36f72314cf1f8c2ef7ea (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4, build-date=2022-05-03T08:36:31.336870, version=8, distribution-scope=public, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, CEPH_POINT_RELEASE=-17.2.0, io.openshift.expose-services=, io.openshift.tags=base centos centos-stream, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, vendor=Red Hat, Inc., release=754, vcs-type=git, io.buildah.version=1.19.8, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., architecture=x86_64, GIT_CLEAN=True, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=HEAD, maintainer=Guillaume Abrioux , com.redhat.license_terms=https://centos.org/legal/licensing-policy/, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, com.redhat.component=centos-stream-container, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, ceph=True, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, name=centos-stream, io.k8s.display-name=CentOS Stream 8) 2024-03-24T11:29:24.358 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:24 smithi139 bash[120132]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4 2024-03-24T11:29:24.608 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:24 smithi139 podman[120149]: 2024-03-24 11:29:24.357870295 +0000 UTC m=+0.543013864 container remove 7568bfa9e74d579ca629d94881a3ee86e263059f2b4b36f72314cf1f8c2ef7ea (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4, io.buildah.version=1.19.8, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, io.openshift.expose-services=, io.openshift.tags=base centos centos-stream, version=8, distribution-scope=public, io.k8s.display-name=CentOS Stream 8, maintainer=Guillaume Abrioux , url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, CEPH_POINT_RELEASE=-17.2.0, GIT_BRANCH=HEAD, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, architecture=x86_64, GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True, name=centos-stream, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, vendor=Red Hat, Inc., vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, GIT_CLEAN=True, build-date=2022-05-03T08:36:31.336870, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, RELEASE=HEAD, vcs-type=git, com.redhat.component=centos-stream-container, release=754) 2024-03-24T11:29:24.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:24 smithi139 ceph-mon[91894]: pgmap v217: 161 pgs: 161 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T11:29:24.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:24 smithi139 ceph-mon[91894]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T11:29:24.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:24 smithi139 ceph-mon[91894]: failed to encode map e149 with expected crc 2024-03-24T11:29:24.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:24 smithi139 ceph-mon[91894]: failed to encode map e149 with expected crc 2024-03-24T11:29:24.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:24 smithi139 ceph-mon[91894]: osdmap e149: 8 total, 7 up, 8 in 2024-03-24T11:29:24.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:24 smithi139 ceph-mon[91894]: failed to encode map e149 with expected crc 2024-03-24T11:29:24.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:24 smithi139 ceph-mon[91894]: failed to encode map e149 with expected crc 2024-03-24T11:29:24.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:24 smithi139 ceph-mon[91894]: failed to encode map e149 with expected crc 2024-03-24T11:29:24.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:24 smithi062 ceph-mon[98643]: pgmap v217: 161 pgs: 161 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T11:29:24.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:24 smithi062 ceph-mon[98643]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T11:29:24.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:24 smithi062 ceph-mon[98643]: failed to encode map e149 with expected crc 2024-03-24T11:29:24.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:24 smithi062 ceph-mon[98643]: failed to encode map e149 with expected crc 2024-03-24T11:29:24.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:24 smithi062 ceph-mon[98643]: osdmap e149: 8 total, 7 up, 8 in 2024-03-24T11:29:24.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:24 smithi062 ceph-mon[98643]: failed to encode map e149 with expected crc 2024-03-24T11:29:24.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:24 smithi062 ceph-mon[98643]: failed to encode map e149 with expected crc 2024-03-24T11:29:24.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:24 smithi062 ceph-mon[98643]: failed to encode map e149 with expected crc 2024-03-24T11:29:24.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:24 smithi062 ceph-mon[90553]: pgmap v217: 161 pgs: 161 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T11:29:24.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:24 smithi062 ceph-mon[90553]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T11:29:24.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:24 smithi062 ceph-mon[90553]: failed to encode map e149 with expected crc 2024-03-24T11:29:24.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:24 smithi062 ceph-mon[90553]: failed to encode map e149 with expected crc 2024-03-24T11:29:24.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:24 smithi062 ceph-mon[90553]: osdmap e149: 8 total, 7 up, 8 in 2024-03-24T11:29:24.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:24 smithi062 ceph-mon[90553]: failed to encode map e149 with expected crc 2024-03-24T11:29:24.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:24 smithi062 ceph-mon[90553]: failed to encode map e149 with expected crc 2024-03-24T11:29:24.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:24 smithi062 ceph-mon[90553]: failed to encode map e149 with expected crc 2024-03-24T11:29:24.986 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:24 smithi139 podman[120241]: 2024-03-24 11:29:24.566399571 +0000 UTC m=+0.017709495 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:29:24.986 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:24 smithi139 podman[120241]: 2024-03-24 11:29:24.692261223 +0000 UTC m=+0.143571116 container create a5f3dc79d356110f0c93d0796c153004750cae36cd052cd20dd14de2e49bbff8 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-db0330b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T11:29:24.986 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:24 smithi139 podman[120241]: 2024-03-24 11:29:24.903094172 +0000 UTC m=+0.354404082 container init a5f3dc79d356110f0c93d0796c153004750cae36cd052cd20dd14de2e49bbff8 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_CLEAN=True, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-24T11:29:24.986 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:24 smithi139 podman[120241]: 2024-03-24 11:29:24.906293522 +0000 UTC m=+0.357603431 container start a5f3dc79d356110f0c93d0796c153004750cae36cd052cd20dd14de2e49bbff8 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, RELEASE=squid-db0330b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-24T11:29:24.986 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:24 smithi139 podman[120241]: 2024-03-24 11:29:24.986383274 +0000 UTC m=+0.437693187 container attach a5f3dc79d356110f0c93d0796c153004750cae36cd052cd20dd14de2e49bbff8 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-db0330b) 2024-03-24T11:29:25.358 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:25 smithi139 podman[120241]: 2024-03-24 11:29:25.195670901 +0000 UTC m=+0.646980792 container died a5f3dc79d356110f0c93d0796c153004750cae36cd052cd20dd14de2e49bbff8 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4-deactivate, RELEASE=squid-db0330b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T11:29:25.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:25 smithi062 ceph-mon[98643]: failed to encode map e150 with expected crc 2024-03-24T11:29:25.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:25 smithi062 ceph-mon[98643]: failed to encode map e150 with expected crc 2024-03-24T11:29:25.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:25 smithi062 ceph-mon[98643]: osdmap e150: 8 total, 7 up, 8 in 2024-03-24T11:29:25.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:25 smithi062 ceph-mon[98643]: failed to encode map e150 with expected crc 2024-03-24T11:29:25.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:25 smithi062 ceph-mon[98643]: failed to encode map e150 with expected crc 2024-03-24T11:29:25.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:25 smithi062 ceph-mon[98643]: failed to encode map e150 with expected crc 2024-03-24T11:29:25.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:25 smithi062 ceph-mon[98643]: failed to encode map e150 with expected crc 2024-03-24T11:29:25.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:25 smithi062 ceph-mon[98643]: failed to encode map e150 with expected crc 2024-03-24T11:29:25.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:25 smithi062 ceph-mon[98643]: failed to encode map e150 with expected crc 2024-03-24T11:29:25.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:25 smithi062 ceph-mon[98643]: failed to encode map e150 with expected crc 2024-03-24T11:29:25.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:25 smithi062 ceph-mon[90553]: failed to encode map e150 with expected crc 2024-03-24T11:29:25.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:25 smithi062 ceph-mon[90553]: failed to encode map e150 with expected crc 2024-03-24T11:29:25.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:25 smithi062 ceph-mon[90553]: osdmap e150: 8 total, 7 up, 8 in 2024-03-24T11:29:25.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:25 smithi062 ceph-mon[90553]: failed to encode map e150 with expected crc 2024-03-24T11:29:25.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:25 smithi062 ceph-mon[90553]: failed to encode map e150 with expected crc 2024-03-24T11:29:25.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:25 smithi062 ceph-mon[90553]: failed to encode map e150 with expected crc 2024-03-24T11:29:25.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:25 smithi062 ceph-mon[90553]: failed to encode map e150 with expected crc 2024-03-24T11:29:25.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:25 smithi062 ceph-mon[90553]: failed to encode map e150 with expected crc 2024-03-24T11:29:25.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:25 smithi062 ceph-mon[90553]: failed to encode map e150 with expected crc 2024-03-24T11:29:25.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:25 smithi062 ceph-mon[90553]: failed to encode map e150 with expected crc 2024-03-24T11:29:25.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:25 smithi139 ceph-mon[91894]: failed to encode map e150 with expected crc 2024-03-24T11:29:25.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:25 smithi139 ceph-mon[91894]: failed to encode map e150 with expected crc 2024-03-24T11:29:25.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:25 smithi139 ceph-mon[91894]: osdmap e150: 8 total, 7 up, 8 in 2024-03-24T11:29:25.823 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:25 smithi139 ceph-mon[91894]: failed to encode map e150 with expected crc 2024-03-24T11:29:25.824 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:25 smithi139 ceph-mon[91894]: failed to encode map e150 with expected crc 2024-03-24T11:29:25.824 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:25 smithi139 ceph-mon[91894]: failed to encode map e150 with expected crc 2024-03-24T11:29:25.824 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:25 smithi139 ceph-mon[91894]: failed to encode map e150 with expected crc 2024-03-24T11:29:25.824 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:25 smithi139 ceph-mon[91894]: failed to encode map e150 with expected crc 2024-03-24T11:29:25.824 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:25 smithi139 ceph-mon[91894]: failed to encode map e150 with expected crc 2024-03-24T11:29:25.824 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:25 smithi139 ceph-mon[91894]: failed to encode map e150 with expected crc 2024-03-24T11:29:26.108 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:25 smithi139 podman[120268]: 2024-03-24 11:29:25.823495684 +0000 UTC m=+0.619668581 container remove a5f3dc79d356110f0c93d0796c153004750cae36cd052cd20dd14de2e49bbff8 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-db0330b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5) 2024-03-24T11:29:26.108 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:25 smithi139 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.4.service: Deactivated successfully. 2024-03-24T11:29:26.108 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:25 smithi139 systemd[1]: Stopped Ceph osd.4 for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:29:26.108 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:25 smithi139 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.4.service: Consumed 10.959s CPU time. 2024-03-24T11:29:26.108 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:26 smithi139 systemd[1]: Starting Ceph osd.4 for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:29:26.544 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:26 smithi139 podman[120375]: 2024-03-24 11:29:26.172803271 +0000 UTC m=+0.018273773 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:29:26.544 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:26 smithi139 podman[120375]: 2024-03-24 11:29:26.299583087 +0000 UTC m=+0.145053574 container create 80b61ca726507e151d98a1430756accb29791df82329afc35e7c21d1688b03c3 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4-activate, RELEASE=squid-db0330b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-24T11:29:26.544 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:26 smithi139 ceph-mon[91894]: pgmap v220: 161 pgs: 19 peering, 15 stale+active+clean, 127 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:29:26.544 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:26 smithi139 ceph-mon[91894]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-03-24T11:29:26.544 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:26.544 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:29:26.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:26 smithi062 ceph-mon[90553]: pgmap v220: 161 pgs: 19 peering, 15 stale+active+clean, 127 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:29:26.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:26 smithi062 ceph-mon[90553]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-03-24T11:29:26.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:26.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:29:26.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:26 smithi062 ceph-mon[98643]: pgmap v220: 161 pgs: 19 peering, 15 stale+active+clean, 127 active+clean; 585 KiB data, 196 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:29:26.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:26 smithi062 ceph-mon[98643]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-03-24T11:29:26.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:26.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:29:26.858 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:26 smithi139 podman[120375]: 2024-03-24 11:29:26.54381426 +0000 UTC m=+0.389284751 container init 80b61ca726507e151d98a1430756accb29791df82329afc35e7c21d1688b03c3 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4-activate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, RELEASE=squid-db0330b, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-24T11:29:26.858 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:26 smithi139 podman[120375]: 2024-03-24 11:29:26.547671518 +0000 UTC m=+0.393142010 container start 80b61ca726507e151d98a1430756accb29791df82329afc35e7c21d1688b03c3 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4-activate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=squid-db0330b, CEPH_POINT_RELEASE=) 2024-03-24T11:29:26.858 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:26 smithi139 podman[120375]: 2024-03-24 11:29:26.627004873 +0000 UTC m=+0.472475380 container attach 80b61ca726507e151d98a1430756accb29791df82329afc35e7c21d1688b03c3 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD) 2024-03-24T11:29:27.358 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:26 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4-activate[120390]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-24T11:29:27.358 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:26 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4-activate[120390]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-24T11:29:27.359 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:26 smithi139 bash[120375]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-24T11:29:27.359 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:26 smithi139 bash[120375]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-24T11:29:27.359 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:26 smithi139 bash[120375]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-24T11:29:27.359 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:26 smithi139 bash[120375]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-24T11:29:27.359 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:26 smithi139 bash[120375]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-4/block 2024-03-24T11:29:27.359 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:26 smithi139 bash[120375]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-24T11:29:27.359 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:26 smithi139 bash[120375]: --> ceph-volume raw activate successful for osd ID: 4 2024-03-24T11:29:27.359 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:26 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4-activate[120390]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-24T11:29:27.359 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:26 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4-activate[120390]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-24T11:29:27.359 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:26 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4-activate[120390]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-4/block 2024-03-24T11:29:27.359 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:26 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4-activate[120390]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-24T11:29:27.359 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:26 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4-activate[120390]: --> ceph-volume raw activate successful for osd ID: 4 2024-03-24T11:29:27.359 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:27 smithi139 podman[120375]: 2024-03-24 11:29:27.012219296 +0000 UTC m=+0.857689779 container died 80b61ca726507e151d98a1430756accb29791df82329afc35e7c21d1688b03c3 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4-activate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-24T11:29:27.730 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:27 smithi139 ceph-mon[91894]: Health check failed: Degraded data redundancy: 61/630 objects degraded (9.683%), 13 pgs degraded (PG_DEGRADED) 2024-03-24T11:29:27.824 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:27 smithi062 ceph-mon[90553]: Health check failed: Degraded data redundancy: 61/630 objects degraded (9.683%), 13 pgs degraded (PG_DEGRADED) 2024-03-24T11:29:27.825 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:27 smithi062 ceph-mon[98643]: Health check failed: Degraded data redundancy: 61/630 objects degraded (9.683%), 13 pgs degraded (PG_DEGRADED) 2024-03-24T11:29:27.982 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:27 smithi139 podman[120507]: 2024-03-24 11:29:27.730826805 +0000 UTC m=+0.712769181 container remove 80b61ca726507e151d98a1430756accb29791df82329afc35e7c21d1688b03c3 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4-activate, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-db0330b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5) 2024-03-24T11:29:27.982 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:27 smithi139 podman[120571]: 2024-03-24 11:29:27.854750458 +0000 UTC m=+0.019593485 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:29:27.982 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:27 smithi139 podman[120571]: 2024-03-24 11:29:27.981934439 +0000 UTC m=+0.146777497 container create c9bfa17037d97001e800fdb375339562dc3b37aaa20dfb0a7e00eb08864d90ea (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True) 2024-03-24T11:29:28.275 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:28 smithi139 podman[120571]: 2024-03-24 11:29:28.200743942 +0000 UTC m=+0.365586964 container init c9bfa17037d97001e800fdb375339562dc3b37aaa20dfb0a7e00eb08864d90ea (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=squid-db0330b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307) 2024-03-24T11:29:28.276 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:28 smithi139 podman[120571]: 2024-03-24 11:29:28.203755233 +0000 UTC m=+0.368598264 container start c9bfa17037d97001e800fdb375339562dc3b37aaa20dfb0a7e00eb08864d90ea (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240307) 2024-03-24T11:29:28.276 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:28 smithi139 ceph-osd[120591]: -- 172.21.15.139:0/3482114531 <== mon.2 v2:172.21.15.139:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556437e271e0 con 0x556437df8c00 2024-03-24T11:29:28.608 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:28 smithi139 bash[120571]: c9bfa17037d97001e800fdb375339562dc3b37aaa20dfb0a7e00eb08864d90ea 2024-03-24T11:29:28.608 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:28 smithi139 systemd[1]: Started Ceph osd.4 for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:29:28.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:28 smithi139 ceph-mon[91894]: pgmap v221: 161 pgs: 13 active+undersized+degraded, 19 peering, 3 stale+active+clean, 27 active+undersized, 99 active+clean; 585 KiB data, 197 MiB used, 715 GiB / 715 GiB avail; 61/630 objects degraded (9.683%) 2024-03-24T11:29:28.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:28.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:28.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:28.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:28.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:28.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:28.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:28.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:28.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:28.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:28.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:28 smithi062 ceph-mon[90553]: pgmap v221: 161 pgs: 13 active+undersized+degraded, 19 peering, 3 stale+active+clean, 27 active+undersized, 99 active+clean; 585 KiB data, 197 MiB used, 715 GiB / 715 GiB avail; 61/630 objects degraded (9.683%) 2024-03-24T11:29:28.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:28.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:28.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:28.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:28.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:28.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:28.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:28.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:28.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:28.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:28.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:28 smithi062 ceph-mon[98643]: pgmap v221: 161 pgs: 13 active+undersized+degraded, 19 peering, 3 stale+active+clean, 27 active+undersized, 99 active+clean; 585 KiB data, 197 MiB used, 715 GiB / 715 GiB avail; 61/630 objects degraded (9.683%) 2024-03-24T11:29:28.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:28.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:28.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:28.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:28.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:28.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:28.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:28.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:28.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:28.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:29.109 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:28 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4[120587]: 2024-03-24T11:29:28.744+0000 7fedc817e7c0 -1 Falling back to public interface 2024-03-24T11:29:29.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:29.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:29.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:29.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:29.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:29.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:29.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:29.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:29.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:29.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:29.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:29.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:29.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:29.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:29.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:29.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:29.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:29.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:29.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:29.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:29.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:29.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:29.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:29.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:29.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:29.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:29.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:29.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:29.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:29.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:29.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:29.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:29.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:29.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:29.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:29.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:29.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:29.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:29.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:29.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:29.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:29.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:29.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:29.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:29.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:29.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:29.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:29.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:29.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:29.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:29.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:29.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:29.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:29.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:29.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:29.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:29.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:29.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:29.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:29.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:29.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:29.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:29.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:29.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:29.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:29.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:29.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:29.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:29.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:29.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:29.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:29.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:29.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:29.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:29.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:29.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:29.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:29.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:29.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:29.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:29.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:29.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:29.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:29.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:29.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:29.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:29.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:29.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:29.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:29.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:29.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:29.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:29.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:29.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:29.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:29.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:29.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:29.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:29.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:29.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:29.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:29.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:29.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:29.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:29.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:29.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:29.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:29.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:29.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:29.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:29.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:29.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:29.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:29.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:29.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:29.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:29.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:29.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:29.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:29.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:29.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:29.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:29.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:29.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:29.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:29.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:29.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:29.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:29.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:29.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:29.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:29.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:29.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:29.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:29.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:29.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:29.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:29.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:29.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:29.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:29.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:29.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:29.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:29.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:29.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:29.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:29.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:29.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:29.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:29.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:29.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:29.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:29.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:29.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:29.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:29.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:29.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:29.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:29.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:29.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:29.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:29.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:29.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:29.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:29.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:29.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:29.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:29.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:29.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:29.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:29.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:29.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:29.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:29.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:29.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:29.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:29.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:29.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:29.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:29.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:29.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:29.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:29.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:29.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:29.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:29.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:29.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:29.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:29.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:29.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:29.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:29.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:29.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:29.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:29.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:29.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:29.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:29.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:29.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:29.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:29.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:29.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:29.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:29.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:30.805 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:30.806 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:30.806 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:30.806 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:30.806 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:30.806 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: pgmap v222: 161 pgs: 24 active+undersized+degraded, 45 active+undersized, 92 active+clean; 585 KiB data, 197 MiB used, 715 GiB / 715 GiB avail; 104/630 objects degraded (16.508%) 2024-03-24T11:29:30.806 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:30.806 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:30.806 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:30.806 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:30.806 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:30.806 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:30.806 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:30.806 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:30.806 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:30.807 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:30.807 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:30.807 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:30.807 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:30.807 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:30.807 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:30.807 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:30.807 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:30.807 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:30.807 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:30.807 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:30.807 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:30.807 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:30.807 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:30.808 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:30.808 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:30.808 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:30.808 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:30.808 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:30.808 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:30.808 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: pgmap v222: 161 pgs: 24 active+undersized+degraded, 45 active+undersized, 92 active+clean; 585 KiB data, 197 MiB used, 715 GiB / 715 GiB avail; 104/630 objects degraded (16.508%) 2024-03-24T11:29:30.808 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:30.808 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:30.808 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:30.808 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:30.809 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:30.809 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:30.809 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:30.809 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:30.809 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:30.809 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:30.809 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:30.809 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:30.809 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:30.809 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:30.809 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:30.809 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:30.809 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:30.809 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:30.809 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:30.810 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:30.810 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:30.810 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:30.810 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:30.810 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:30.810 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:30.810 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:30.810 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:30.810 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:30.810 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:30.811 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:30.811 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:30.811 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:30.811 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:30.811 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:30.811 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:30.811 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:30.811 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:30.811 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:30.811 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:30.811 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:30.811 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:30.811 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:30.811 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:30.811 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:30.812 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:30.812 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:30.812 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:30.812 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:30.812 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:30.812 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:30.812 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:30.812 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:30.812 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:30.812 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:30.812 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:30.812 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:30.813 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:30.813 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:30.813 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:30.813 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:30.813 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:30.813 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:30.813 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:30.813 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:30.813 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:30.813 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:30.813 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:30.813 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:30.813 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:30.813 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:30.813 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:30.814 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:30.814 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:30.814 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:30.814 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:30.814 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:30.814 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:30.814 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:30.814 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:30.814 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:30.814 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:30.814 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:30.815 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:30.815 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:30.815 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:30.815 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:30.815 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:30.815 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:30.815 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-03-24T11:29:30.815 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:30.815 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:30.815 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:30.815 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:30.815 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:30.816 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:30.816 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:30.816 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:30.816 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:30.816 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:30.816 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:30.816 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-03-24T11:29:30.816 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:30.816 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:30.816 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:30.816 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:30.816 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:30.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:30.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:30.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:30.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:30.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:30.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: pgmap v222: 161 pgs: 24 active+undersized+degraded, 45 active+undersized, 92 active+clean; 585 KiB data, 197 MiB used, 715 GiB / 715 GiB avail; 104/630 objects degraded (16.508%) 2024-03-24T11:29:30.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:30.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:30.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:30.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:30.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:30.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:30.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:30.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:30.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:30.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:30.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:30.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:30.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:30.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:30.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:30.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:30.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:30.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:30.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:30.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:30.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:30.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:30.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:30.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:30.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:30.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:30.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:30.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:30.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:30.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:30.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:30.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:30.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:30.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:30.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:30.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:30.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:30.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:30.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:30.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:30.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:30.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:30.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:30.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:30.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:30.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:30.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:30.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:30.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:30.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:30.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:30.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:30.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:30.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:30.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:30.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:30.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:30.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:30.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:30.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-03-24T11:29:30.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:30.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:30.864 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:30.864 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:30.864 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:31.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:29:30 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:29:30] "GET /metrics HTTP/1.1" 200 38584 "" "Prometheus/2.33.4" 2024-03-24T11:29:31.858 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4[120587]: 2024-03-24T11:29:31.506+0000 7fedc817e7c0 -1 osd.4 0 read_superblock omap replica is missing. 2024-03-24T11:29:31.858 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4[120587]: 2024-03-24T11:29:31.544+0000 7fedc817e7c0 -1 osd.4 148 log_to_monitors true 2024-03-24T11:29:31.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:31.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:31.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:31.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:31.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:31.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:31.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:31.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:31.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:31.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:31.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:31.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:31.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:31.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:31.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:31.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:31.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:31.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:31.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:31.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:31.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:31.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:31.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:31.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:31.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:31.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:31.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:31.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:31.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:31.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:31.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:31.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:31.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:31.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:31.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:31.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:31.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:31.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:31.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:31.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:31.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:31.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:31.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:31.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:31.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:31.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:31.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:31.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:31.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:31.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:31.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:31.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:31.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:31.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:31.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:31.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:31.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:31.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:31.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:31.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:31.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:31.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:31.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:31.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:31.864 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:31.864 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:31.864 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:31.864 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:31.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:31.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:31.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:31.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:31.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:31.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:31.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:31.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:31.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:31.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:31.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:31.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:31.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:31.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:31.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:31.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:31.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:31.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:31.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:31.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:31.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:31.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:31.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:31.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:31.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:31.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:31.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:31.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:31.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:31.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:31.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:31.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:31.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:31.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:31.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:31.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:31.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:31.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:31.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:31.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:31.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:31.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:31.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:31.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:31.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:31.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:31.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:31.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:31.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:31.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:31.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:31.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:31.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:31.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:31.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:31.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:31.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:31.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:31.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:31.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:31.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:31.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:31.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:31.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:31.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:31.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:31.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:31.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:31.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:31.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:31.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:31.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:31.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:31.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:31.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:31.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:31.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:31.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:31.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:31.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:31.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:31.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:31.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:31.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:31.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:31.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:31.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:31.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:31.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:31.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:31.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:31.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:31.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:31.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:31.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:31.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:31.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:31.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:31.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:31.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:31.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:31.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:31.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:31.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:31.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:31.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:31.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:31.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:31.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:31.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:31.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:31.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:31.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:31.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:31.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:31.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:31.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:31.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:31.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:31.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:31.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:31.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:31.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:31.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:31.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:31.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:31.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:31.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:31.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:31.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:31.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:31.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:31.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:31.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:31.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:31.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:32.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:32.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:32.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:32.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:32.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: pgmap v223: 161 pgs: 24 active+undersized+degraded, 45 active+undersized, 92 active+clean; 585 KiB data, 197 MiB used, 715 GiB / 715 GiB avail; 104/630 objects degraded (16.508%) 2024-03-24T11:29:32.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:32.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:32.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:32.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:32.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:32.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:32.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='osd.4 [v2:172.21.15.139:6800/105222727,v1:172.21.15.139:6801/105222727]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-24T11:29:32.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-24T11:29:32.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:32.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:32.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:32.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:32.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:32.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:32.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:32.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:32.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:32.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:32.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:32.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:32.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:32.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:32.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:32.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:32.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:32.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:32.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:32.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: pgmap v223: 161 pgs: 24 active+undersized+degraded, 45 active+undersized, 92 active+clean; 585 KiB data, 197 MiB used, 715 GiB / 715 GiB avail; 104/630 objects degraded (16.508%) 2024-03-24T11:29:32.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:32.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:32.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:32.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:32.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:32.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:32.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='osd.4 [v2:172.21.15.139:6800/105222727,v1:172.21.15.139:6801/105222727]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-24T11:29:32.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-24T11:29:32.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:32.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:32.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:32.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:32.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:32.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:32.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:32.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:32.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:32.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:32.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:32.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:32.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:32.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:32.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:32.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:32.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:32.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:32.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:32.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:32.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:32.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:32.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:32.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:32.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:32.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:32.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:32.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:32.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:32.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:32.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:32.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:32.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:32.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:32.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:32.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:32.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:32.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:32.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:32.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:32.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:32.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:32.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:32.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:32.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:32.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:32.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:32.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:32.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:32.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:32.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:32.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:32.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:32.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:32.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:32.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:32.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:32.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:32.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:32.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:32.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:32.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:32.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:32.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:32.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:32.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:32.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:32.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:32.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:32.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:32.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:32.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:32.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:32.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:32.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:32.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:32.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:32.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:32.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:32.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:32.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:32.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:32.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:32.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:32.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:32.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:32.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:32.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:32.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:32.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:32.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:32.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:32.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:32.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:32.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:32.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:32.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:32.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:32.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:32.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:32.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:32.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:32.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:32.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:32.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:32.858 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4[120587]: 2024-03-24T11:29:32.583+0000 7fedb95d6700 -1 osd.4 148 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T11:29:32.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:32.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:32.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:32.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:32.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: pgmap v223: 161 pgs: 24 active+undersized+degraded, 45 active+undersized, 92 active+clean; 585 KiB data, 197 MiB used, 715 GiB / 715 GiB avail; 104/630 objects degraded (16.508%) 2024-03-24T11:29:32.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:32.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:32.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:32.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:32.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:32.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:32.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='osd.4 [v2:172.21.15.139:6800/105222727,v1:172.21.15.139:6801/105222727]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-24T11:29:32.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-24T11:29:32.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:32.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:32.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:32.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:32.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:32.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:32.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:32.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:32.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:32.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:32.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:32.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:32.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:32.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:32.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:32.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:32.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:32.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:32.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:32.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:32.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:32.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:32.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:32.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:32.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:32.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:32.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:32.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:32.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:32.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:32.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:32.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:32.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:32.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:32.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:32.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:32.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:32.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:32.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:32.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:32.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:32.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:32.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:32.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:32.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:32.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:32.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:32.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:32.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:32.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:32.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:32.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:32.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:32.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:32.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:32.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:32.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:32.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:32.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:32.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:33.602 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:33.603 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:33.603 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:33.603 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:33.603 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:29:33.603 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:33.603 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:33.603 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-24T11:29:33.603 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: failed to encode map e151 with expected crc 2024-03-24T11:29:33.603 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: osdmap e151: 8 total, 7 up, 8 in 2024-03-24T11:29:33.603 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='osd.4 [v2:172.21.15.139:6800/105222727,v1:172.21.15.139:6801/105222727]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:29:33.603 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:29:33.603 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:33.603 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:33.604 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:33.604 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:33.604 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:33.604 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:33.604 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:33.604 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:33.604 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:33.604 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:33.604 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:33.604 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:33.604 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:33.604 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:33.604 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:33.604 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:33.604 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:33.605 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:33.605 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:33.605 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:33.605 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:33.605 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:33.605 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:33.605 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:33.605 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:33.605 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:33.605 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:33.605 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:33.605 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:33.605 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:33.605 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:33.606 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:33.606 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:33.606 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:33.606 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:33.606 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:33.606 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:33.606 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:33.606 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:33.606 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:33.606 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:33.606 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:33.606 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:33.606 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:33.607 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:33.607 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:33.607 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:33.607 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:33.607 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:33.607 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:33.607 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:33.607 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:33.607 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:33.607 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:33.607 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:33.607 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:33.607 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:33.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:33.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:33.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:33.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:33.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:33.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:33.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:33.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:29:33.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:33.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:33.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-24T11:29:33.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: failed to encode map e151 with expected crc 2024-03-24T11:29:33.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: osdmap e151: 8 total, 7 up, 8 in 2024-03-24T11:29:33.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='osd.4 [v2:172.21.15.139:6800/105222727,v1:172.21.15.139:6801/105222727]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:29:33.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:29:33.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:33.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:33.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:33.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:33.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:33.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:33.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:33.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:33.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:33.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:33.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:33.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:33.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:33.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:33.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:33.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:33.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:33.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:33.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:33.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:33.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:33.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:33.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:29:33.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:33.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:33.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-24T11:29:33.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: failed to encode map e151 with expected crc 2024-03-24T11:29:33.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: osdmap e151: 8 total, 7 up, 8 in 2024-03-24T11:29:33.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='osd.4 [v2:172.21.15.139:6800/105222727,v1:172.21.15.139:6801/105222727]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:29:33.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:29:33.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:33.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:33.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:33.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:33.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:33.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:33.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:33.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:33.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:33.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:33.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:33.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:33.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:33.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:33.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:33.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:33.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:33.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:33.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:33.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:33.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:33.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:33.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:33.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:33.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:33.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:33.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:33.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:33.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:33.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:33.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:33.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:33.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:33.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:33.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:33.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:33.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:33.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:33.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:33.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:33.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:33.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:33.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:33.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:33.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:33.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:33.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:33.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:33.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:33.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:33.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:33.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:33.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:33.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:33.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:33.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:33.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:33.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:33.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:33.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:33.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:33.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:33.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:33.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:33.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:33.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:33.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:33.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:33.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:33.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:33.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:33.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:33.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:33.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:33.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:33.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:33.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:33.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:33.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:33.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:33.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:33.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:33.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:33.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:33.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:33.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:33.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:33.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:33.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:33.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:33.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:33.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:33.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:33.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:33.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:33.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:33.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:33.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:33.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:33.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:33.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:33.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:34.751 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:34.751 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:34.751 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:34.751 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:34.751 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:34.751 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: pgmap v225: 161 pgs: 24 active+undersized+degraded, 45 active+undersized, 92 active+clean; 585 KiB data, 207 MiB used, 715 GiB / 715 GiB avail; 228 B/s rd, 0 op/s; 104/630 objects degraded (16.508%) 2024-03-24T11:29:34.751 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:34.751 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T11:29:34.752 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: failed to encode map e151 with expected crc 2024-03-24T11:29:34.752 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: osd.4 [v2:172.21.15.139:6800/105222727,v1:172.21.15.139:6801/105222727] boot 2024-03-24T11:29:34.752 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: osdmap e152: 8 total, 8 up, 8 in 2024-03-24T11:29:34.752 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: failed to encode map e152 with expected crc 2024-03-24T11:29:34.752 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T11:29:34.752 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: failed to encode map e151 with expected crc 2024-03-24T11:29:34.752 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: failed to encode map e151 with expected crc 2024-03-24T11:29:34.752 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: failed to encode map e151 with expected crc 2024-03-24T11:29:34.752 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:34.752 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:34.752 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:34.752 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:34.752 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:34.753 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:34.753 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:34.753 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:34.753 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:34.753 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:34.753 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:34.753 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:34.753 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:34.753 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:34.753 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:34.753 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:34.754 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:34.754 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:34.754 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:34.754 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:34.754 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:34.754 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:34.754 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:34.754 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:34.754 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:34.754 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:34.754 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:34.754 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:34.754 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:34.754 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:34.755 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:34.755 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:34.755 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:34.755 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:34.755 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:34.755 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:34.755 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:34.755 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:34.755 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:34.755 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:34.755 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:34.755 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:34.756 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:34.756 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:34.756 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:34.756 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: Health check update: Degraded data redundancy: 104/630 objects degraded (16.508%), 24 pgs degraded (PG_DEGRADED) 2024-03-24T11:29:34.756 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:34.756 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:34.756 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:34.756 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:34.756 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:34.756 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:34.756 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:34.756 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:34.756 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:34.757 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:34.757 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:34.757 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:34.757 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:34.757 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:34.757 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:34.757 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:34.757 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:34.757 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:34.757 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:34.757 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:34.757 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:34 smithi139 ceph-mon[91894]: osdmap e153: 8 total, 8 up, 8 in 2024-03-24T11:29:34.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:34.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:34.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:34.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:34.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:34.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: pgmap v225: 161 pgs: 24 active+undersized+degraded, 45 active+undersized, 92 active+clean; 585 KiB data, 207 MiB used, 715 GiB / 715 GiB avail; 228 B/s rd, 0 op/s; 104/630 objects degraded (16.508%) 2024-03-24T11:29:34.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:34.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T11:29:34.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: failed to encode map e151 with expected crc 2024-03-24T11:29:34.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: osd.4 [v2:172.21.15.139:6800/105222727,v1:172.21.15.139:6801/105222727] boot 2024-03-24T11:29:34.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: osdmap e152: 8 total, 8 up, 8 in 2024-03-24T11:29:34.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: failed to encode map e152 with expected crc 2024-03-24T11:29:34.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T11:29:34.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: failed to encode map e151 with expected crc 2024-03-24T11:29:34.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: failed to encode map e151 with expected crc 2024-03-24T11:29:34.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: failed to encode map e151 with expected crc 2024-03-24T11:29:34.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:34.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:34.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:34.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:34.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:34.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:34.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:34.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:34.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:34.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:34.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:34.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:34.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:34.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:34.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:34.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:34.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:34.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:34.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:34.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:34.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:34.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:34.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:34.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:34.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:34.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:34.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:34.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:34.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:34.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:34.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:34.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:34.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:34.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:34.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:34.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:34.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:34.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:34.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:34.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:34.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:34.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:34.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:34.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:34.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:34.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:34.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:34.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:34.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:34.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:34.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: pgmap v225: 161 pgs: 24 active+undersized+degraded, 45 active+undersized, 92 active+clean; 585 KiB data, 207 MiB used, 715 GiB / 715 GiB avail; 228 B/s rd, 0 op/s; 104/630 objects degraded (16.508%) 2024-03-24T11:29:34.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:34.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T11:29:34.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: failed to encode map e151 with expected crc 2024-03-24T11:29:34.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: osd.4 [v2:172.21.15.139:6800/105222727,v1:172.21.15.139:6801/105222727] boot 2024-03-24T11:29:34.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: osdmap e152: 8 total, 8 up, 8 in 2024-03-24T11:29:34.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: failed to encode map e152 with expected crc 2024-03-24T11:29:34.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T11:29:34.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: failed to encode map e151 with expected crc 2024-03-24T11:29:34.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: failed to encode map e151 with expected crc 2024-03-24T11:29:34.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: failed to encode map e151 with expected crc 2024-03-24T11:29:34.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:34.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:34.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:34.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:34.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:34.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:34.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:34.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:34.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:34.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:34.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:34.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:34.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:34.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:34.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:34.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:34.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: Health check update: Degraded data redundancy: 104/630 objects degraded (16.508%), 24 pgs degraded (PG_DEGRADED) 2024-03-24T11:29:34.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:34.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:34.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:34.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:34.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:34.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:34.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:34.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:34.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:34.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:34.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:34.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:34.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:34.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:34.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:34.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:34.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:34.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:34.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:34.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:34.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[90553]: osdmap e153: 8 total, 8 up, 8 in 2024-03-24T11:29:34.989 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:34.989 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:34.989 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:34.989 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:34.989 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:34.989 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:34.989 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:34.989 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:34.989 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:34.990 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:34.990 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:34.990 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:34.990 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:34.990 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:34.990 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:34.990 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:34.990 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:34.991 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:34.991 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:34.991 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:34.991 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:34.991 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:34.991 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:34.991 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:34.991 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:34.991 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:34.991 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:34.991 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:34.991 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:34.991 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: Health check update: Degraded data redundancy: 104/630 objects degraded (16.508%), 24 pgs degraded (PG_DEGRADED) 2024-03-24T11:29:34.992 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:34.992 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:34.992 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:34.992 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:34.992 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:34.992 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:34.992 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:34.992 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:34.992 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:34.993 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:34.993 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:34.993 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:34.993 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:34.993 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:34.993 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:34.993 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:34.993 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:34.993 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:34.994 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:34.994 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:34.994 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:34 smithi062 ceph-mon[98643]: osdmap e153: 8 total, 8 up, 8 in 2024-03-24T11:29:35.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:35.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:35.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:35.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:35.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:35.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:35.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:35.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: failed to encode map e153 with expected crc 2024-03-24T11:29:35.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: failed to encode map e153 with expected crc 2024-03-24T11:29:35.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: failed to encode map e153 with expected crc 2024-03-24T11:29:35.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: failed to encode map e153 with expected crc 2024-03-24T11:29:35.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: failed to encode map e153 with expected crc 2024-03-24T11:29:35.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: failed to encode map e153 with expected crc 2024-03-24T11:29:35.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: failed to encode map e153 with expected crc 2024-03-24T11:29:35.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:35.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:35.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:35.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:35.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:35.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:35.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:35.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:35.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:35.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:35.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:35.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:35.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:35.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:35.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:35.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:35.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:35.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:35.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:35.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:35.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:35.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:35.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:35.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:35.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:35.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:35.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:35.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:35.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:35.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:35.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:35.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:35.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:35.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:35.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:35.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:35.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:35.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:35.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:35.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:35.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:35.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:35.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:35.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:35.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:35.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:35.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:35.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:35.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:35.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:35.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:35.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:35.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:35.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:35.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:35.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:35.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:35.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:35.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:35.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:35.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:35.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: failed to encode map e153 with expected crc 2024-03-24T11:29:35.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: failed to encode map e153 with expected crc 2024-03-24T11:29:35.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: failed to encode map e153 with expected crc 2024-03-24T11:29:35.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: failed to encode map e153 with expected crc 2024-03-24T11:29:35.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: failed to encode map e153 with expected crc 2024-03-24T11:29:35.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: failed to encode map e153 with expected crc 2024-03-24T11:29:35.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: failed to encode map e153 with expected crc 2024-03-24T11:29:35.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:35.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:35.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:35.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:35.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:35.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:35.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:35.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:35.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:35.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:35.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:35.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:35.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:35.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:35.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:35.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:35.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:35.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:35.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:35.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:35.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:35.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:35.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:35.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:35.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:35.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:35.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: failed to encode map e153 with expected crc 2024-03-24T11:29:35.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: failed to encode map e153 with expected crc 2024-03-24T11:29:35.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: failed to encode map e153 with expected crc 2024-03-24T11:29:35.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: failed to encode map e153 with expected crc 2024-03-24T11:29:35.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: failed to encode map e153 with expected crc 2024-03-24T11:29:35.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: failed to encode map e153 with expected crc 2024-03-24T11:29:35.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: failed to encode map e153 with expected crc 2024-03-24T11:29:35.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:35.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:35.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:35.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:35.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:35.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:35.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:35.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:35.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:35.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:35.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:35.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:35.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:35.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:35.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:35.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:35.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:35.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:35.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:35.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:35.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:35.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:35.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:35.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:35.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:35.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:35.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:35.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:35.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:35.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:35.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:35.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:35.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:35.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:35.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:35.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:35.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:35.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:35.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:35.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:35.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:35.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:35.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:35.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:35.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:35.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:35.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:35.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:35.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:35.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:35.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:35.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:35.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:35.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:35.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:35.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:35.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:35.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:35.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:35.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:35.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:35.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:35.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:35.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:35.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:35.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:35.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:35.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:35.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:35.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:35.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:35.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:35.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:35.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:35.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:35.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:35.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:35.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:35.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:35.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:35.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:35.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:35.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:35.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:35.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:35.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:35.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:35.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:35.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:36.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:36.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:36.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:36.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:36.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:36.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: pgmap v228: 161 pgs: 11 peering, 20 active+undersized+degraded, 38 active+undersized, 92 active+clean; 585 KiB data, 207 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 85/630 objects degraded (13.492%) 2024-03-24T11:29:36.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:36.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:36.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:36.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:36.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:36.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:36.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:36.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:36.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:36.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:36.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:36.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:36.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:36.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:36.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:36.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:36.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:36.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:36.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:36.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:36.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:36.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:36.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:36.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:36.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:36.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:36.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:36.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:36.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:36.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:36.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:36.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:36.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:36.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:36.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:36.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:36.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:36.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:36.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:36.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:36.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:36.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:36.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:36.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:36.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:36.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:36.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:36.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:36.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:36.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:36.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:36.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:36.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:36.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:36.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:36.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:36.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:36.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:36.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:36.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:36.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:36.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:36.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:36.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:36.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:36.864 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:36.864 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:36.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:36.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:36.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:36.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:36.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:36.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: pgmap v228: 161 pgs: 11 peering, 20 active+undersized+degraded, 38 active+undersized, 92 active+clean; 585 KiB data, 207 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 85/630 objects degraded (13.492%) 2024-03-24T11:29:36.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:36.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:36.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:36.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:36.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:36.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:36.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:36.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:36.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:36.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:36.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:36.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:36.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:36.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:36.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:36.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:36.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:36.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:36.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:36.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:36.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:36.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:36.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:36.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:36.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:36.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:36.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:36.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:36.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: pgmap v228: 161 pgs: 11 peering, 20 active+undersized+degraded, 38 active+undersized, 92 active+clean; 585 KiB data, 207 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 85/630 objects degraded (13.492%) 2024-03-24T11:29:36.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:36.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:36.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:36.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:36.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:36.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:36.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:36.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:36.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:36.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:36.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:36.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:36.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:36.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:36.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:36.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:36.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:36.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:36.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:36.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:36.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:36.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:36.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:36.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:36.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:36.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:36.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:36.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:36.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:36.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:36.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:36.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:36.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:36.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:36.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:36.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:36.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:36.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:36.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:36.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:36.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:36.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:36.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:36.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:36.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:36.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:36.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:36.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:36.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:36.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:36.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:36.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:36.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:36.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:36.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:36.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:36.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:36.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:36.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:36.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:36.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:36.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:36.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:36.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:36.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:36.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:36.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:36.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:36.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:36.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:36.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:36.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:36.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:36.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:36.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:36.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:36.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:36.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:36.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:36.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:36.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:36.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:36.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:36.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:36.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:36.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:36.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:36.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:36.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:36.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:36.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:36.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:36.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:36.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:36.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:36.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:36.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:36.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:36.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:36.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:36.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:36.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:36.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:36.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:36.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:36.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:36.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:36.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:36.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:36.989 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:36.989 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:37.848 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:37.848 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:37.849 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:37.849 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:37.849 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:37.849 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:37.849 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:37.849 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:37.849 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:37.849 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:37.849 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:37.849 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:37.849 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:37.849 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:37.849 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:37.850 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:37.850 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:37.850 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:37.850 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:37.850 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:37.850 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:37.850 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:37.850 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:37.850 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:37.850 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:37.850 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:37.850 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:37.850 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:37.850 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:37.851 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:37.851 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:37.851 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:37.851 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:37.851 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:37.851 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:37.851 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:37.852 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:37.852 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:37.852 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:37.852 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:37.852 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:37.852 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:37.852 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:37.852 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:37.852 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:37.852 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:37.852 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:37.852 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:37.853 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:37.853 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:37.853 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:37.853 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:37.853 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:37.853 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:37.853 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:37.853 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:37.853 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:37.853 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:37.853 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:37.853 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:37.853 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:37.853 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:37.854 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:37.854 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:37.854 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:37.854 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:37.888 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:37.889 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:37.889 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:37.889 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:37.889 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:37.889 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:37.889 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:37.889 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:37.889 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:37.889 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:37.889 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:37.890 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:37.890 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:37.890 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:37.890 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:37.890 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:37.890 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:37.890 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:37.890 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:37.890 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:37.890 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:37.890 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:37.891 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:37.891 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:37.891 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:37.891 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:37.891 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:37.891 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:37.891 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:37.891 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:37.891 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:37.892 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:37.892 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:37.892 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:37.892 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:37.892 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:37.892 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:37.892 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:37.892 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:37.892 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:37.892 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:37.892 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:37.892 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:37.893 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:37.893 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:37.893 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:37.893 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:37.893 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:37.893 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:37.893 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:37.893 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:37.893 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:37.893 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:37.893 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:37.894 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:37.894 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:37.894 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:37.894 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:37.894 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:37.894 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:37.894 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:37.895 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:37.895 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:37.895 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:37.895 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:37.895 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:37.895 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:37.895 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:37.895 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:37.895 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:37.895 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:37.896 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:37.896 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:37.896 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:37.896 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:37.896 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:37.896 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:37.896 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:37.896 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:37.896 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:37.896 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:37.896 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:37.897 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:37.897 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:37.897 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:37.897 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:37.897 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:37.897 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:37.897 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:37.897 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:37.897 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:37.898 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:37.898 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:37.898 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:37.898 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:37.898 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:37.898 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:37.898 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:37.898 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:37.898 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:37.898 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:37.898 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:37.899 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:37.899 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:37.899 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:37.899 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:37.899 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:37.899 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:37.899 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:37.899 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:37.899 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:37.899 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:37.899 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:37.899 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:37.900 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:37.900 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:37.900 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:37.900 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:37.900 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:37.900 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:37.900 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:37.900 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:37.901 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:37.901 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:37.901 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:37.901 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:37.901 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:37.901 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:37.901 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:37.901 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:37.901 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:37.902 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:38.038 INFO:teuthology.orchestra.run.smithi062.stdout:true 2024-03-24T11:29:38.778 INFO:teuthology.orchestra.run.smithi062.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T11:29:38.778 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi062 smithi062 running 9s ago 19m - - 2024-03-24T11:29:38.778 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi139 smithi139 running 18s ago 18m - - 2024-03-24T11:29:38.778 INFO:teuthology.orchestra.run.smithi062.stdout:alertmanager.a smithi062 *:9093,9094 running (8m) 9s ago 13m 25.1M - 0.23.0 ba2b418f427c 818298d58140 2024-03-24T11:29:38.778 INFO:teuthology.orchestra.run.smithi062.stdout:grafana.a smithi139 *:3000 running (8m) 18s ago 13m 53.6M - 8.3.5 dad864ee21e9 b48167e7171e 2024-03-24T11:29:38.778 INFO:teuthology.orchestra.run.smithi062.stdout:iscsi.foo.smithi062.dggsvk smithi062 running (7m) 9s ago 12m 64.1M - 3.5 e1d6a67b021e 0a98806ec839 2024-03-24T11:29:38.778 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.x smithi139 *:8443,9283,8765 running (6m) 18s ago 18m 452M - 19.0.0-1667-gdb0330b1 4f325f8440e9 d94987b11387 2024-03-24T11:29:38.779 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.y smithi062 *:8443,9283,8765 running (7m) 9s ago 19m 529M - 19.0.0-1667-gdb0330b1 4f325f8440e9 ca1a49028d65 2024-03-24T11:29:38.779 INFO:teuthology.orchestra.run.smithi062.stdout:mon.a smithi062 running (5m) 9s ago 20m 66.0M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 f7afe6e28c88 2024-03-24T11:29:38.779 INFO:teuthology.orchestra.run.smithi062.stdout:mon.b smithi139 running (4m) 18s ago 18m 46.9M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 ca357beea4dd 2024-03-24T11:29:38.779 INFO:teuthology.orchestra.run.smithi062.stdout:mon.c smithi062 running (5m) 9s ago 18m 56.7M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 459e7449ec39 2024-03-24T11:29:38.779 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.a smithi062 *:9100 running (9m) 9s ago 14m 16.8M - 1.3.1 1dbe0e931976 79d5a02a999f 2024-03-24T11:29:38.779 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.b smithi139 *:9100 running (8m) 18s ago 14m 15.8M - 1.3.1 1dbe0e931976 8f9f4608e465 2024-03-24T11:29:38.779 INFO:teuthology.orchestra.run.smithi062.stdout:osd.0 smithi062 running (3m) 9s ago 17m 57.8M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 dfe4920b49c4 2024-03-24T11:29:38.779 INFO:teuthology.orchestra.run.smithi062.stdout:osd.1 smithi062 running (2m) 9s ago 17m 48.9M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 c443b3fe97c1 2024-03-24T11:29:38.779 INFO:teuthology.orchestra.run.smithi062.stdout:osd.2 smithi062 running (102s) 9s ago 17m 44.7M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 0eba40dc96b3 2024-03-24T11:29:38.779 INFO:teuthology.orchestra.run.smithi062.stdout:osd.3 smithi062 running (40s) 9s ago 16m 53.2M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 7501d2e4c718 2024-03-24T11:29:38.779 INFO:teuthology.orchestra.run.smithi062.stdout:osd.4 smithi139 starting - - - 4096M 2024-03-24T11:29:38.779 INFO:teuthology.orchestra.run.smithi062.stdout:osd.5 smithi139 running (16m) 18s ago 16m 63.1M 4096M 17.2.0 e1d6a67b021e dd0abd599a14 2024-03-24T11:29:38.779 INFO:teuthology.orchestra.run.smithi062.stdout:osd.6 smithi139 running (15m) 18s ago 15m 61.2M 4096M 17.2.0 e1d6a67b021e a4b1289aca69 2024-03-24T11:29:38.779 INFO:teuthology.orchestra.run.smithi062.stdout:osd.7 smithi139 running (15m) 18s ago 15m 64.5M 4096M 17.2.0 e1d6a67b021e bc7fadbc1384 2024-03-24T11:29:38.779 INFO:teuthology.orchestra.run.smithi062.stdout:prometheus.a smithi139 *:9095 running (7m) 18s ago 14m 71.2M - 2.33.4 514e6a882f6e 2ed47f810d33 2024-03-24T11:29:38.780 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi062.sacpds smithi062 *:8000 running (12m) 9s ago 12m 74.7M - 17.2.0 e1d6a67b021e 89f4ac308e98 2024-03-24T11:29:38.780 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi139.yljesh smithi139 *:8000 running (12m) 18s ago 12m 73.8M - 17.2.0 e1d6a67b021e e189bdc12cae 2024-03-24T11:29:38.780 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi062.xvpxnk smithi062 *:80 running (12m) 9s ago 12m 74.5M - 17.2.0 e1d6a67b021e d1aa74adab44 2024-03-24T11:29:38.780 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi139.pbtrmp smithi139 *:80 running (12m) 18s ago 12m 77.6M - 17.2.0 e1d6a67b021e 7f8e336755d7 2024-03-24T11:29:38.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:38.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:38.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:38.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:38.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:38.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: pgmap v229: 161 pgs: 11 peering, 12 active+undersized+degraded, 18 active+undersized, 120 active+clean; 585 KiB data, 207 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 37/630 objects degraded (5.873%) 2024-03-24T11:29:38.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:38.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:38.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:38.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:38.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:38.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:38.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:38.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:38.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:38.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:38.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:38.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:38.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:38.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:38.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:38.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:38.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:38.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:38.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:38.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:38.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:38.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:38.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:38.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:38.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:38.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: from='client.64480 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:29:38.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:38.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:38.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:38.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:38.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:38.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:38.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:38.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:38.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:38.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:38.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:38.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:38.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:38.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:38.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:38.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:38.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:38.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:38.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:38.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:38.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:38.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:38.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:38.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:38.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:38.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: pgmap v229: 161 pgs: 11 peering, 12 active+undersized+degraded, 18 active+undersized, 120 active+clean; 585 KiB data, 207 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 37/630 objects degraded (5.873%) 2024-03-24T11:29:38.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:38.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:38.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:38.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:38.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:38.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:38.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:38.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:38.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:38.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:38.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:38.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:38.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:38.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:38.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:38.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:38.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:38.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:38.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:38.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:38.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:38.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:38.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:38.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:38.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:38.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:38.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:38.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:38.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: pgmap v229: 161 pgs: 11 peering, 12 active+undersized+degraded, 18 active+undersized, 120 active+clean; 585 KiB data, 207 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 37/630 objects degraded (5.873%) 2024-03-24T11:29:38.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:38.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:38.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:38.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:38.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:38.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:38.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:38.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:38.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:38.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:38.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:38.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:38.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:38.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:38.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:38.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:38.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:38.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:38.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:38.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:38.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:38.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:38.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:38.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:38.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:38.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: from='client.64480 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:29:38.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:38.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:38.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:38.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:38.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:38.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:38.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:38.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:38.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:38.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:38.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:38.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:38.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:38.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:38.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:38.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:38.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:38.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:38.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:38.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:38.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:38.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:38.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: from='client.64480 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:29:38.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:38.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:38.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:38.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:38.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:38.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:38.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:38.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:38.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:38.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:38.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:38.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:38.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:38.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:38.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:38.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:38.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:38.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:38.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:38.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:39.208 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:29:39.208 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": { 2024-03-24T11:29:39.208 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 3 2024-03-24T11:29:39.208 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:29:39.208 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": { 2024-03-24T11:29:39.208 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-24T11:29:39.208 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:29:39.209 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": { 2024-03-24T11:29:39.209 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 3, 2024-03-24T11:29:39.209 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 5 2024-03-24T11:29:39.209 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:29:39.209 INFO:teuthology.orchestra.run.smithi062.stdout: "rgw": { 2024-03-24T11:29:39.209 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4 2024-03-24T11:29:39.209 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:29:39.209 INFO:teuthology.orchestra.run.smithi062.stdout: "overall": { 2024-03-24T11:29:39.209 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 7, 2024-03-24T11:29:39.209 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 10 2024-03-24T11:29:39.209 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-24T11:29:39.209 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:29:39.562 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:29:39.563 INFO:teuthology.orchestra.run.smithi062.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69", 2024-03-24T11:29:39.563 INFO:teuthology.orchestra.run.smithi062.stdout: "in_progress": true, 2024-03-24T11:29:39.563 INFO:teuthology.orchestra.run.smithi062.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T11:29:39.563 INFO:teuthology.orchestra.run.smithi062.stdout: "services_complete": [ 2024-03-24T11:29:39.563 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr", 2024-03-24T11:29:39.563 INFO:teuthology.orchestra.run.smithi062.stdout: "mon" 2024-03-24T11:29:39.563 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-03-24T11:29:39.563 INFO:teuthology.orchestra.run.smithi062.stdout: "progress": "9/23 daemons upgraded", 2024-03-24T11:29:39.563 INFO:teuthology.orchestra.run.smithi062.stdout: "message": "Currently upgrading osd daemons", 2024-03-24T11:29:39.563 INFO:teuthology.orchestra.run.smithi062.stdout: "is_paused": false 2024-03-24T11:29:39.563 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:29:39.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:39.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:39.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:39.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: from='client.74357 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:29:39.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:39.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:39.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:39.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:39.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:39.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:39.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:39.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: from='client.44523 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:29:39.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:39.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:39.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:39.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:39.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:39.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:39.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:39.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:39.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:39.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:39.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:39.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:39.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:39.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:39.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:39.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:39.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:39.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:39.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:39.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:39.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:39.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:39.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:39.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: from='client.? 172.21.15.62:0/615769673' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:39.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:39.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:39.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:39.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:39.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:39.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: Health check update: Degraded data redundancy: 37/630 objects degraded (5.873%), 12 pgs degraded (PG_DEGRADED) 2024-03-24T11:29:39.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:39.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:39.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:39.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:39.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:39.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:39.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:39.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:39.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:39.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:39.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:39.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:39.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:39.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:39.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:39.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:39.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: from='client.74357 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:29:39.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:39.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:39.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:39.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:39.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:39.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:39.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:39.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: from='client.44523 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:29:39.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:39.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:39.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:39.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:39.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:39.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:39.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:39.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:39.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:39.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:39.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:39.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:39.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:39.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:39.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:39.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:39.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:39.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:39.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:39.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:39.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:39.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: from='client.74357 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:29:39.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:39.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:39.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:39.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:39.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:39.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:39.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:39.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: from='client.44523 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:29:39.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:39.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:39.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:39.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:39.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:39.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:39.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:39.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:39.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:39.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:39.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:39.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:39.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:39.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:39.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:39.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:39.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:39.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:39.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:39.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:39.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:39.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:39.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:39.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: from='client.? 172.21.15.62:0/615769673' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:39.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:39.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:39.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:39.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:39.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:39.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: Health check update: Degraded data redundancy: 37/630 objects degraded (5.873%), 12 pgs degraded (PG_DEGRADED) 2024-03-24T11:29:39.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:39.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:39.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:39.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:39.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:39.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:39.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:39.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:39.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:39.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:39.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:39.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:39.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:39.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:39.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:39.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:39.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:39.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:39.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: from='client.? 172.21.15.62:0/615769673' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:39.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:39.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:39.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:39.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:39.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:39.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: Health check update: Degraded data redundancy: 37/630 objects degraded (5.873%), 12 pgs degraded (PG_DEGRADED) 2024-03-24T11:29:39.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:39.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:39.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:39.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:39.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:39.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:39.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:39.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:39.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:39.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:39.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:39.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:39.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:39.992 INFO:teuthology.orchestra.run.smithi062.stdout:HEALTH_WARN Degraded data redundancy: 37/630 objects degraded (5.873%), 12 pgs degraded 2024-03-24T11:29:39.992 INFO:teuthology.orchestra.run.smithi062.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 37/630 objects degraded (5.873%), 12 pgs degraded 2024-03-24T11:29:39.992 INFO:teuthology.orchestra.run.smithi062.stdout: pg 2.f is active+undersized+degraded, acting [0,7] 2024-03-24T11:29:39.992 INFO:teuthology.orchestra.run.smithi062.stdout: pg 3.3 is active+undersized+degraded, acting [0,6] 2024-03-24T11:29:39.992 INFO:teuthology.orchestra.run.smithi062.stdout: pg 3.9 is active+undersized+degraded, acting [2,7] 2024-03-24T11:29:39.992 INFO:teuthology.orchestra.run.smithi062.stdout: pg 3.14 is active+undersized+degraded, acting [7,6] 2024-03-24T11:29:39.992 INFO:teuthology.orchestra.run.smithi062.stdout: pg 3.19 is active+undersized+degraded, acting [1,3] 2024-03-24T11:29:39.992 INFO:teuthology.orchestra.run.smithi062.stdout: pg 3.1a is active+undersized+degraded, acting [1,2] 2024-03-24T11:29:39.992 INFO:teuthology.orchestra.run.smithi062.stdout: pg 4.9 is active+undersized+degraded, acting [1,3] 2024-03-24T11:29:39.992 INFO:teuthology.orchestra.run.smithi062.stdout: pg 4.d is active+undersized+degraded, acting [2,1] 2024-03-24T11:29:39.992 INFO:teuthology.orchestra.run.smithi062.stdout: pg 4.e is active+undersized+degraded, acting [6,1] 2024-03-24T11:29:39.993 INFO:teuthology.orchestra.run.smithi062.stdout: pg 5.1c is active+undersized+degraded, acting [3,2] 2024-03-24T11:29:39.993 INFO:teuthology.orchestra.run.smithi062.stdout: pg 5.1d is active+undersized+degraded, acting [1,0] 2024-03-24T11:29:39.993 INFO:teuthology.orchestra.run.smithi062.stdout: pg 6.1a is active+undersized+degraded, acting [5,1] 2024-03-24T11:29:40.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:40.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:40.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:40.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: pgmap v230: 161 pgs: 161 active+clean; 585 KiB data, 211 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-03-24T11:29:40.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:40.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='client.44535 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:29:40.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:40.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:40.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:40.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:40.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:40.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:40.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:40.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:40.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:40.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:40.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:40.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:40.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:40.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:40.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:40.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:40.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:40.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:40.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:40.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:40.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:40.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:40.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:40.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:40.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:40.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:40.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='client.? 172.21.15.62:0/1933343160' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:29:40.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:40.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:40.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:40.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:40.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:40.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:40.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:40.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:40.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:40.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:40.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:40.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:40.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:40.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:40.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:40.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:40.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:40.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:40.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:40.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:40.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:40.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 37/630 objects degraded (5.873%), 12 pgs degraded) 2024-03-24T11:29:40.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: Cluster is now healthy 2024-03-24T11:29:40.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:40.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:40.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:40.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:40.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:40.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:40.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:40.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:40.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:40.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:40.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:40.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:40.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-24T11:29:40.877 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:29:40] "GET /metrics HTTP/1.1" 200 38584 "" "Prometheus/2.33.4" 2024-03-24T11:29:40.877 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:40.878 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:40.878 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:40.878 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: pgmap v230: 161 pgs: 161 active+clean; 585 KiB data, 211 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-03-24T11:29:40.878 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:40.878 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='client.44535 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:29:40.878 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:40.878 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:40.878 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:40.878 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:40.878 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:40.878 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:40.878 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:40.879 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:40.879 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:40.879 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:40.879 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:40.879 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:40.879 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:40.879 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:40.879 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:40.879 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:40.879 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:40.879 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:40.879 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:40.879 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:40.880 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:40.880 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:40.880 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:40.880 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:40.880 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:40.880 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:40.880 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:40.880 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: pgmap v230: 161 pgs: 161 active+clean; 585 KiB data, 211 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-03-24T11:29:40.880 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:40.880 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='client.44535 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:29:40.880 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:40.881 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:40.881 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:40.881 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:40.881 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:40.881 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:40.881 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:40.881 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:40.881 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:40.881 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:40.881 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:40.881 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:40.881 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:40.881 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:40.882 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:40.882 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:40.882 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:40.882 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:40.882 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:40.882 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:40.882 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:40.882 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:40.882 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:40.882 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:40.883 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:40.883 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:40.883 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='client.? 172.21.15.62:0/1933343160' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:29:40.883 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:40.883 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:40.883 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:40.883 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:40.884 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:40.884 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:40.884 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:40.884 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:40.884 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:40.884 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:40.884 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:40.884 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:40.884 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:40.884 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:40.884 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:40.884 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:40.884 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:40.884 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:40.884 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:40.885 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:40.885 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:40.885 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 37/630 objects degraded (5.873%), 12 pgs degraded) 2024-03-24T11:29:40.885 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: Cluster is now healthy 2024-03-24T11:29:40.885 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:40.885 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:40.885 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:40.885 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:40.885 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:40.885 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:40.885 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:40.885 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:40.886 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:40.886 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:40.886 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:40.886 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:40.886 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-24T11:29:40.886 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:40.886 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:40.886 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='client.? 172.21.15.62:0/1933343160' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:29:40.886 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:40.886 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:40.887 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:40.887 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:40.887 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:40.887 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:40.887 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:40.887 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:40.887 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:40.887 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:40.887 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:40.887 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:40.887 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:40.887 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:40.887 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:40.887 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:40.887 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:40.888 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:40.888 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:40.888 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:40.888 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:40.888 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 37/630 objects degraded (5.873%), 12 pgs degraded) 2024-03-24T11:29:40.888 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: Cluster is now healthy 2024-03-24T11:29:40.888 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:40.888 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:40.888 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:40.888 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:40.888 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:40.888 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:40.888 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:40.888 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:40.888 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:40.888 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:40.889 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:40.889 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:40.889 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-24T11:29:41.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:41 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:41.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:41 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:41.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:41 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:41.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:41 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:41.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:41 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:41.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:41 smithi139 ceph-mon[91894]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-24T11:29:41.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:41 smithi139 ceph-mon[91894]: Upgrade: osd.4 is safe to restart 2024-03-24T11:29:41.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:41.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:29:41.909 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:41 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:41.909 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:41 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:41.909 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:41 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:41.909 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:41 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:41.909 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:41 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:41.909 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:41 smithi062 ceph-mon[90553]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-24T11:29:41.909 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:41 smithi062 ceph-mon[90553]: Upgrade: osd.4 is safe to restart 2024-03-24T11:29:41.909 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:41.909 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:29:41.909 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:41 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:41.910 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:41 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:41.910 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:41 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:41.910 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:41 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:41.910 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:41 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:41.910 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:41 smithi062 ceph-mon[98643]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-24T11:29:41.910 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:41 smithi062 ceph-mon[98643]: Upgrade: osd.4 is safe to restart 2024-03-24T11:29:41.910 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:41.910 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:29:43.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:43 smithi062 ceph-mon[90553]: pgmap v231: 161 pgs: 161 active+clean; 585 KiB data, 211 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:29:43.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:43.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:43.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:43.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:43.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:43.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:43.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:43 smithi062 ceph-mon[98643]: pgmap v231: 161 pgs: 161 active+clean; 585 KiB data, 211 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:29:43.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:43.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:43.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:43.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:43.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:43.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:43.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:43 smithi139 ceph-mon[91894]: pgmap v231: 161 pgs: 161 active+clean; 585 KiB data, 211 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:29:43.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:43.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:43.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:43.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:43.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:43.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:44.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:44 smithi139 ceph-mon[91894]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:29:44.409 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:44 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:29:44.410 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:44 smithi062 ceph-mon[98643]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:29:45.154 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:45 smithi139 ceph-mon[91894]: pgmap v232: 161 pgs: 161 active+clean; 585 KiB data, 211 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-24T11:29:45.154 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-24T11:29:45.154 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:45.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:45 smithi062 ceph-mon[90553]: pgmap v232: 161 pgs: 161 active+clean; 585 KiB data, 211 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-24T11:29:45.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-24T11:29:45.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:45.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:45 smithi062 ceph-mon[98643]: pgmap v232: 161 pgs: 161 active+clean; 585 KiB data, 211 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-24T11:29:45.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-24T11:29:45.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:46.292 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:46 smithi139 ceph-mon[91894]: Upgrade: Updating osd.4 2024-03-24T11:29:46.292 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:46 smithi139 ceph-mon[91894]: Deploying daemon osd.4 on smithi139 2024-03-24T11:29:46.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:46 smithi062 ceph-mon[98643]: Upgrade: Updating osd.4 2024-03-24T11:29:46.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:46 smithi062 ceph-mon[98643]: Deploying daemon osd.4 on smithi139 2024-03-24T11:29:46.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:46 smithi062 ceph-mon[90553]: Upgrade: Updating osd.4 2024-03-24T11:29:46.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:46 smithi062 ceph-mon[90553]: Deploying daemon osd.4 on smithi139 2024-03-24T11:29:47.143 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:47 smithi139 ceph-mon[91894]: pgmap v233: 161 pgs: 161 active+clean; 585 KiB data, 211 MiB used, 715 GiB / 715 GiB avail; 940 B/s rd, 0 op/s 2024-03-24T11:29:47.143 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:47.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:47 smithi062 ceph-mon[90553]: pgmap v233: 161 pgs: 161 active+clean; 585 KiB data, 211 MiB used, 715 GiB / 715 GiB avail; 940 B/s rd, 0 op/s 2024-03-24T11:29:47.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:47.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:47 smithi062 ceph-mon[98643]: pgmap v233: 161 pgs: 161 active+clean; 585 KiB data, 211 MiB used, 715 GiB / 715 GiB avail; 940 B/s rd, 0 op/s 2024-03-24T11:29:47.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:49.213 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:48 smithi139 systemd[1]: Stopping Ceph osd.4 for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:29:49.213 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:49 smithi139 ceph-mon[91894]: pgmap v234: 161 pgs: 161 active+clean; 585 KiB data, 211 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:29:49.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:49 smithi062 ceph-mon[90553]: pgmap v234: 161 pgs: 161 active+clean; 585 KiB data, 211 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:29:49.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:49 smithi062 ceph-mon[98643]: pgmap v234: 161 pgs: 161 active+clean; 585 KiB data, 211 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:29:49.608 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:49 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4[120587]: 2024-03-24T11:29:49.322+0000 7fedbe7e5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T11:29:49.608 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:49 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4[120587]: 2024-03-24T11:29:49.322+0000 7fedbe7e5700 -1 osd.4 153 *** Got signal Terminated *** 2024-03-24T11:29:49.608 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:49 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4[120587]: 2024-03-24T11:29:49.322+0000 7fedbe7e5700 -1 osd.4 153 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T11:29:50.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:50 smithi062 ceph-mon[90553]: osd.4 marked itself down and dead 2024-03-24T11:29:50.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:50 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:50.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:50 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:50.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:50 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:50.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:50 smithi062 ceph-mon[98643]: osd.4 marked itself down and dead 2024-03-24T11:29:50.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:50 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:50.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:50 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:50.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:50 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:50.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:50 smithi139 ceph-mon[91894]: osd.4 marked itself down and dead 2024-03-24T11:29:50.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:50 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:50.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:50 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:50.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:50 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:50.848 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:50 smithi139 podman[126903]: 2024-03-24 11:29:50.543745914 +0000 UTC m=+1.411807524 container died c9bfa17037d97001e800fdb375339562dc3b37aaa20dfb0a7e00eb08864d90ea (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-db0330b, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T11:29:51.060 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:29:50 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:29:50] "GET /metrics HTTP/1.1" 200 38593 "" "Prometheus/2.33.4" 2024-03-24T11:29:51.108 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:51 smithi139 podman[126903]: 2024-03-24 11:29:51.06423981 +0000 UTC m=+1.932301419 container cleanup c9bfa17037d97001e800fdb375339562dc3b37aaa20dfb0a7e00eb08864d90ea (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-db0330b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-24T11:29:51.108 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:51 smithi139 bash[126903]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4 2024-03-24T11:29:51.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:51 smithi139 ceph-mon[91894]: pgmap v235: 161 pgs: 161 active+clean; 585 KiB data, 211 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:29:51.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:51 smithi139 ceph-mon[91894]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T11:29:51.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:51 smithi139 ceph-mon[91894]: failed to encode map e154 with expected crc 2024-03-24T11:29:51.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:51 smithi139 ceph-mon[91894]: failed to encode map e154 with expected crc 2024-03-24T11:29:51.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:51 smithi139 ceph-mon[91894]: osdmap e154: 8 total, 7 up, 8 in 2024-03-24T11:29:51.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:51 smithi139 ceph-mon[91894]: failed to encode map e154 with expected crc 2024-03-24T11:29:51.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:51 smithi139 ceph-mon[91894]: failed to encode map e154 with expected crc 2024-03-24T11:29:51.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:51 smithi139 ceph-mon[91894]: failed to encode map e154 with expected crc 2024-03-24T11:29:51.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:51 smithi139 ceph-mon[91894]: failed to encode map e154 with expected crc 2024-03-24T11:29:51.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:51 smithi139 ceph-mon[91894]: failed to encode map e154 with expected crc 2024-03-24T11:29:51.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:51 smithi062 ceph-mon[90553]: pgmap v235: 161 pgs: 161 active+clean; 585 KiB data, 211 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:29:51.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:51 smithi062 ceph-mon[90553]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T11:29:51.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:51 smithi062 ceph-mon[90553]: failed to encode map e154 with expected crc 2024-03-24T11:29:51.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:51 smithi062 ceph-mon[90553]: failed to encode map e154 with expected crc 2024-03-24T11:29:51.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:51 smithi062 ceph-mon[90553]: osdmap e154: 8 total, 7 up, 8 in 2024-03-24T11:29:51.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:51 smithi062 ceph-mon[90553]: failed to encode map e154 with expected crc 2024-03-24T11:29:51.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:51 smithi062 ceph-mon[90553]: failed to encode map e154 with expected crc 2024-03-24T11:29:51.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:51 smithi062 ceph-mon[90553]: failed to encode map e154 with expected crc 2024-03-24T11:29:51.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:51 smithi062 ceph-mon[90553]: failed to encode map e154 with expected crc 2024-03-24T11:29:51.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:51 smithi062 ceph-mon[90553]: failed to encode map e154 with expected crc 2024-03-24T11:29:51.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:51 smithi062 ceph-mon[98643]: pgmap v235: 161 pgs: 161 active+clean; 585 KiB data, 211 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:29:51.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:51 smithi062 ceph-mon[98643]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T11:29:51.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:51 smithi062 ceph-mon[98643]: failed to encode map e154 with expected crc 2024-03-24T11:29:51.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:51 smithi062 ceph-mon[98643]: failed to encode map e154 with expected crc 2024-03-24T11:29:51.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:51 smithi062 ceph-mon[98643]: osdmap e154: 8 total, 7 up, 8 in 2024-03-24T11:29:51.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:51 smithi062 ceph-mon[98643]: failed to encode map e154 with expected crc 2024-03-24T11:29:51.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:51 smithi062 ceph-mon[98643]: failed to encode map e154 with expected crc 2024-03-24T11:29:51.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:51 smithi062 ceph-mon[98643]: failed to encode map e154 with expected crc 2024-03-24T11:29:51.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:51 smithi062 ceph-mon[98643]: failed to encode map e154 with expected crc 2024-03-24T11:29:51.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:51 smithi062 ceph-mon[98643]: failed to encode map e154 with expected crc 2024-03-24T11:29:51.830 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:51 smithi139 podman[126932]: 2024-03-24 11:29:51.382938472 +0000 UTC m=+0.832918613 container remove c9bfa17037d97001e800fdb375339562dc3b37aaa20dfb0a7e00eb08864d90ea (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-24T11:29:52.094 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:51 smithi139 podman[127024]: 2024-03-24 11:29:51.734120086 +0000 UTC m=+0.018875242 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:29:52.094 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:51 smithi139 podman[127024]: 2024-03-24 11:29:51.842214645 +0000 UTC m=+0.126969782 container create bbf48473851e3d6cd84c0c077d93cc6eff838cbe9d43a88de5e3fe2c629bd5bf (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, RELEASE=squid-db0330b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-24T11:29:52.094 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:52 smithi139 ceph-mon[91894]: failed to encode map e155 with expected crc 2024-03-24T11:29:52.095 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:52 smithi139 ceph-mon[91894]: failed to encode map e155 with expected crc 2024-03-24T11:29:52.095 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:52 smithi139 ceph-mon[91894]: failed to encode map e155 with expected crc 2024-03-24T11:29:52.095 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:52 smithi139 ceph-mon[91894]: osdmap e155: 8 total, 7 up, 8 in 2024-03-24T11:29:52.095 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:52 smithi139 ceph-mon[91894]: failed to encode map e155 with expected crc 2024-03-24T11:29:52.095 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:52 smithi139 ceph-mon[91894]: pgmap v238: 161 pgs: 2 active+undersized+degraded, 20 stale+active+clean, 4 active+undersized, 135 active+clean; 585 KiB data, 211 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 6/630 objects degraded (0.952%) 2024-03-24T11:29:52.358 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:52 smithi139 podman[127024]: 2024-03-24 11:29:52.094586903 +0000 UTC m=+0.379342049 container init bbf48473851e3d6cd84c0c077d93cc6eff838cbe9d43a88de5e3fe2c629bd5bf (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-db0330b, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-24T11:29:52.358 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:52 smithi139 podman[127024]: 2024-03-24 11:29:52.097783157 +0000 UTC m=+0.382538303 container start bbf48473851e3d6cd84c0c077d93cc6eff838cbe9d43a88de5e3fe2c629bd5bf (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-db0330b, GIT_CLEAN=True, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-24T11:29:52.358 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:52 smithi139 podman[127024]: 2024-03-24 11:29:52.202972692 +0000 UTC m=+0.487727843 container attach bbf48473851e3d6cd84c0c077d93cc6eff838cbe9d43a88de5e3fe2c629bd5bf (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-db0330b, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-24T11:29:52.378 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:52 smithi062 ceph-mon[90553]: failed to encode map e155 with expected crc 2024-03-24T11:29:52.379 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:52 smithi062 ceph-mon[90553]: failed to encode map e155 with expected crc 2024-03-24T11:29:52.379 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:52 smithi062 ceph-mon[90553]: failed to encode map e155 with expected crc 2024-03-24T11:29:52.379 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:52 smithi062 ceph-mon[90553]: osdmap e155: 8 total, 7 up, 8 in 2024-03-24T11:29:52.379 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:52 smithi062 ceph-mon[90553]: failed to encode map e155 with expected crc 2024-03-24T11:29:52.379 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:52 smithi062 ceph-mon[90553]: pgmap v238: 161 pgs: 2 active+undersized+degraded, 20 stale+active+clean, 4 active+undersized, 135 active+clean; 585 KiB data, 211 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 6/630 objects degraded (0.952%) 2024-03-24T11:29:52.379 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:52 smithi062 ceph-mon[98643]: failed to encode map e155 with expected crc 2024-03-24T11:29:52.379 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:52 smithi062 ceph-mon[98643]: failed to encode map e155 with expected crc 2024-03-24T11:29:52.379 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:52 smithi062 ceph-mon[98643]: failed to encode map e155 with expected crc 2024-03-24T11:29:52.379 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:52 smithi062 ceph-mon[98643]: osdmap e155: 8 total, 7 up, 8 in 2024-03-24T11:29:52.379 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:52 smithi062 ceph-mon[98643]: failed to encode map e155 with expected crc 2024-03-24T11:29:52.379 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:52 smithi062 ceph-mon[98643]: pgmap v238: 161 pgs: 2 active+undersized+degraded, 20 stale+active+clean, 4 active+undersized, 135 active+clean; 585 KiB data, 211 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 6/630 objects degraded (0.952%) 2024-03-24T11:29:52.845 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:52 smithi139 podman[127024]: 2024-03-24 11:29:52.39209325 +0000 UTC m=+0.676848391 container died bbf48473851e3d6cd84c0c077d93cc6eff838cbe9d43a88de5e3fe2c629bd5bf (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-db0330b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-24T11:29:53.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:53 smithi139 ceph-mon[91894]: Health check failed: Degraded data redundancy: 6/630 objects degraded (0.952%), 2 pgs degraded (PG_DEGRADED) 2024-03-24T11:29:53.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:53 smithi139 ceph-mon[91894]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:29:53.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:53 smithi062 ceph-mon[90553]: Health check failed: Degraded data redundancy: 6/630 objects degraded (0.952%), 2 pgs degraded (PG_DEGRADED) 2024-03-24T11:29:53.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:53 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:29:53.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:53 smithi062 ceph-mon[98643]: Health check failed: Degraded data redundancy: 6/630 objects degraded (0.952%), 2 pgs degraded (PG_DEGRADED) 2024-03-24T11:29:53.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:53 smithi062 ceph-mon[98643]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:29:53.608 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:53 smithi139 podman[127051]: 2024-03-24 11:29:53.240462736 +0000 UTC m=+0.843332435 container remove bbf48473851e3d6cd84c0c077d93cc6eff838cbe9d43a88de5e3fe2c629bd5bf (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True) 2024-03-24T11:29:53.608 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:53 smithi139 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.4.service: Deactivated successfully. 2024-03-24T11:29:53.608 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:53 smithi139 systemd[1]: Stopped Ceph osd.4 for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:29:53.608 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:53 smithi139 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.4.service: Consumed 1.914s CPU time. 2024-03-24T11:29:53.608 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:53 smithi139 systemd[1]: Starting Ceph osd.4 for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:29:53.952 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:53 smithi139 podman[127154]: 2024-03-24 11:29:53.595987275 +0000 UTC m=+0.018273903 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:29:53.952 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:53 smithi139 podman[127154]: 2024-03-24 11:29:53.733324418 +0000 UTC m=+0.155611031 container create 5fc5e8b8d8c1d32b1feb60cf0f5aab08dbb3ef8a4ab6e4aacee172ada3f14c5a (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4-activate, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T11:29:53.952 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:53 smithi139 podman[127154]: 2024-03-24 11:29:53.952162744 +0000 UTC m=+0.374449357 container init 5fc5e8b8d8c1d32b1feb60cf0f5aab08dbb3ef8a4ab6e4aacee172ada3f14c5a (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4-activate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-24T11:29:54.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:54 smithi139 ceph-mon[91894]: pgmap v239: 161 pgs: 17 active+undersized+degraded, 8 stale+active+clean, 33 active+undersized, 103 active+clean; 585 KiB data, 212 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 74/630 objects degraded (11.746%) 2024-03-24T11:29:54.358 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:53 smithi139 podman[127154]: 2024-03-24 11:29:53.95535435 +0000 UTC m=+0.377640977 container start 5fc5e8b8d8c1d32b1feb60cf0f5aab08dbb3ef8a4ab6e4aacee172ada3f14c5a (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4-activate, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-db0330b, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T11:29:54.358 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:54 smithi139 podman[127154]: 2024-03-24 11:29:54.052188627 +0000 UTC m=+0.474475248 container attach 5fc5e8b8d8c1d32b1feb60cf0f5aab08dbb3ef8a4ab6e4aacee172ada3f14c5a (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4-activate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-db0330b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240307) 2024-03-24T11:29:54.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:54 smithi062 ceph-mon[90553]: pgmap v239: 161 pgs: 17 active+undersized+degraded, 8 stale+active+clean, 33 active+undersized, 103 active+clean; 585 KiB data, 212 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 74/630 objects degraded (11.746%) 2024-03-24T11:29:54.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:54 smithi062 ceph-mon[98643]: pgmap v239: 161 pgs: 17 active+undersized+degraded, 8 stale+active+clean, 33 active+undersized, 103 active+clean; 585 KiB data, 212 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 74/630 objects degraded (11.746%) 2024-03-24T11:29:54.731 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:54 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4-activate[127169]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-24T11:29:54.731 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:54 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4-activate[127169]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-24T11:29:54.731 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:54 smithi139 bash[127154]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-24T11:29:54.731 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:54 smithi139 bash[127154]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-24T11:29:54.731 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:54 smithi139 bash[127154]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-24T11:29:54.731 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:54 smithi139 bash[127154]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-24T11:29:54.732 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:54 smithi139 bash[127154]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-4/block 2024-03-24T11:29:54.732 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:54 smithi139 bash[127154]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-24T11:29:54.732 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:54 smithi139 bash[127154]: --> ceph-volume raw activate successful for osd ID: 4 2024-03-24T11:29:54.732 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:54 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4-activate[127169]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-24T11:29:54.732 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:54 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4-activate[127169]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-24T11:29:54.732 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:54 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4-activate[127169]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-4/block 2024-03-24T11:29:54.732 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:54 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4-activate[127169]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-24T11:29:54.732 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:54 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4-activate[127169]: --> ceph-volume raw activate successful for osd ID: 4 2024-03-24T11:29:54.732 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:54 smithi139 podman[127154]: 2024-03-24 11:29:54.423341873 +0000 UTC m=+0.845628479 container died 5fc5e8b8d8c1d32b1feb60cf0f5aab08dbb3ef8a4ab6e4aacee172ada3f14c5a (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4-activate, CEPH_POINT_RELEASE=, RELEASE=squid-db0330b, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T11:29:55.358 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:55 smithi139 podman[127301]: 2024-03-24 11:29:55.039616691 +0000 UTC m=+0.610351442 container remove 5fc5e8b8d8c1d32b1feb60cf0f5aab08dbb3ef8a4ab6e4aacee172ada3f14c5a (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4-activate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-db0330b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-24T11:29:55.358 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:55 smithi139 podman[127363]: 2024-03-24 11:29:55.158043595 +0000 UTC m=+0.017742404 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:29:55.358 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:55 smithi139 podman[127363]: 2024-03-24 11:29:55.273925223 +0000 UTC m=+0.133624066 container create 5d0f7f05ec6d71fd9afc6d855efcf50fcde6399b260f23dada7ef82d4bd36f58 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4, org.label-schema.build-date=20240307, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-db0330b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-24T11:29:55.854 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:55 smithi139 podman[127363]: 2024-03-24 11:29:55.526676426 +0000 UTC m=+0.386375239 container init 5d0f7f05ec6d71fd9afc6d855efcf50fcde6399b260f23dada7ef82d4bd36f58 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-db0330b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0) 2024-03-24T11:29:55.854 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:55 smithi139 podman[127363]: 2024-03-24 11:29:55.529926237 +0000 UTC m=+0.389625043 container start 5d0f7f05ec6d71fd9afc6d855efcf50fcde6399b260f23dada7ef82d4bd36f58 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-db0330b, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-03-24T11:29:55.854 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:55 smithi139 ceph-osd[127382]: -- 172.21.15.139:0/3005810777 <== mon.2 v2:172.21.15.139:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564ced6b91e0 con 0x564ced68ac00 2024-03-24T11:29:55.854 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:55 smithi139 bash[127363]: 5d0f7f05ec6d71fd9afc6d855efcf50fcde6399b260f23dada7ef82d4bd36f58 2024-03-24T11:29:55.854 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:55 smithi139 systemd[1]: Started Ceph osd.4 for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:29:56.108 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4[127378]: 2024-03-24T11:29:56.090+0000 7f09274a07c0 -1 Falling back to public interface 2024-03-24T11:29:56.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: pgmap v240: 161 pgs: 24 active+undersized+degraded, 45 active+undersized, 92 active+clean; 585 KiB data, 212 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 104/630 objects degraded (16.508%) 2024-03-24T11:29:56.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:56.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:56.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:56.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:56.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:56.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:56.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:56.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:56.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:56.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:56.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:56.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:56.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:56.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:56.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:56.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:56.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:56.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:56.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:56.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:56.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:56.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:56.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:56.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:56.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:56.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:56.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:56.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: pgmap v240: 161 pgs: 24 active+undersized+degraded, 45 active+undersized, 92 active+clean; 585 KiB data, 212 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 104/630 objects degraded (16.508%) 2024-03-24T11:29:56.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:56.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:56.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:56.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:56.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:56.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:56.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:56.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:56.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:56.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:56.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:56.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:56.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:56.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:56.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:56.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:56.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:56.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:56.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:56.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:56.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:56.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:56.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:56.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:56.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:56.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:56.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:56.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:56.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:56.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:56.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:29:56.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:56.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:56.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:56.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:56.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:56.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:56.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:56.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:56.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:56.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:56.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:56.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:56.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:56.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:56.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:56.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:56.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:56.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:56.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:56.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:56.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:56.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:56.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:56.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:56.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:56.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:56.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:56.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:56.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:29:56.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:56.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:56.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:56.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:56.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:56.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:56.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:56.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:56.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:56.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:56.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:56.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:56.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:56.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:56.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:56.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:56.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:56.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:56.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:56.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:56.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:56.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:56.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:56.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:56.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:56.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:56.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:56.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:56.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:56.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:56.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:56.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:56.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:56.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:56.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:57.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: pgmap v240: 161 pgs: 24 active+undersized+degraded, 45 active+undersized, 92 active+clean; 585 KiB data, 212 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 104/630 objects degraded (16.508%) 2024-03-24T11:29:57.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:57.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:57.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:57.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:57.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:57.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:57.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:57.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:57.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:57.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:57.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:57.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:57.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:57.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:57.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:57.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:57.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:57.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:57.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:57.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:57.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:57.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:57.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:57.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:57.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:57.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:57.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:57.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:57.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:57.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:57.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:29:57.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:57.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:57.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:57.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:57.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:57.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:57.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:57.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:57.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:57.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:57.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:57.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:57.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:57.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:57.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:57.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:57.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:57.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:57.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:57.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:57.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:57.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:57.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:57.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:57.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:57.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:57.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:57.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:57.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:57.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:57.936 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:57.936 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:57.936 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:57.936 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:57.936 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:57.936 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:57.936 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:57.936 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:57.936 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:57.936 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:57.937 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:57.937 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:57.937 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:57.937 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:57.937 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:57.937 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:57.937 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:57.937 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:57.937 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:57.937 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:57.937 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:57.937 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:57.937 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:57.937 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:57.937 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:57.938 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:57.938 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:57.938 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:57.938 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:57.938 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:57.938 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:57.938 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:57.938 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:57.938 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:57.938 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:57.938 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:57.938 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:57.938 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:57.938 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:57.938 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:57.939 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:57.939 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:57.939 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:57.939 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:57.939 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:57.939 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:57.939 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:57.939 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:57.939 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:57.939 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:57.939 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:57.939 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:57.939 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:57.939 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:57.940 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:57.940 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:57.940 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:57.940 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:57.940 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:57.940 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:57.940 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:57.940 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:57.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:57.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:57.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:57.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:57.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:57.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:57.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:57.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:57.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:57.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:57.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:57.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:57.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:57.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:57.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:57.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:57.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:57.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:57.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:57.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:57.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:57.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:57.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:57.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:57.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:57.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:57.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:57.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:57.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:57.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:57.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:57.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:57.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:57.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:57.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:57.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:57.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:57.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:57.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:57.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:57.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:57.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:57.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:57.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:57.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:57.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:57.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:57.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:57.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:57.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:57.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:57.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:57.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:57.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:57.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:57.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:57.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:57.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:57.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:57.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:57.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:57.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:57.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:57.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:57.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:57.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:57.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:57.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:57.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:57.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:57.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:57.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:57.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:57.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:57.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:57.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:57.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:57.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:57.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:57.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:57.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:57.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:57.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:57.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:57.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:57.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:57.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:57.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:57.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:57.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:57.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:57.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:57.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:57.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:57.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:57.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:57.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:57.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:57.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:57.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:57.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:57.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:57.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:57.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:57.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:57.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:57.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:57.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:57.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:57.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:57.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:57.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:57.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:57.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:57.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:57.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:57.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:57.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:57.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:57.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:57.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:57.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:57.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:57.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:58.943 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:29:58 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4[127378]: 2024-03-24T11:29:58.869+0000 7f09274a07c0 -1 osd.4 153 log_to_monitors true 2024-03-24T11:29:58.944 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:58 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:58.944 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:58 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:58.944 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:58 smithi139 ceph-mon[91894]: pgmap v241: 161 pgs: 24 active+undersized+degraded, 45 active+undersized, 92 active+clean; 585 KiB data, 212 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 104/630 objects degraded (16.508%) 2024-03-24T11:29:58.944 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:58 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:58.944 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:58 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:58.944 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:58 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:58.944 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:58.944 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:58.944 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:58.944 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:58.944 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:58.944 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:58 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:58.945 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:58.945 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:58.945 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:58.945 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:58.945 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:58.945 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:58 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:58.945 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:58.945 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:58.945 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:58.945 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:58.945 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:58.945 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:58 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:58.946 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:58.946 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:58.946 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:58.946 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:58.946 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:58.946 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:58.946 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:58.946 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:58.946 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:58.946 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:58.947 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:58.947 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:58.947 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:58.947 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:58.947 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:58.947 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:58.947 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:58.947 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:58.947 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:58.947 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:58.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:58.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:58.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[98643]: pgmap v241: 161 pgs: 24 active+undersized+degraded, 45 active+undersized, 92 active+clean; 585 KiB data, 212 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 104/630 objects degraded (16.508%) 2024-03-24T11:29:58.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:58.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:58.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:58.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:58.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:58.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:58.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:58.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:58.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:58.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:58.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:58.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:58.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:58.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:58.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:58.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:58.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:58.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:58.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:58.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:58.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:58.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:58.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:58.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:58.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:58.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:58.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:58.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:58.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:58.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:58.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:58.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:58.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:58.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:58.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:58.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:58.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:58.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:58.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:58.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:58.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:58.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:58.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:58.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[90553]: pgmap v241: 161 pgs: 24 active+undersized+degraded, 45 active+undersized, 92 active+clean; 585 KiB data, 212 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 104/630 objects degraded (16.508%) 2024-03-24T11:29:58.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:58.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:58.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:58.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:58.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:58.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:58.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:58.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:58.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:58.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:58.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:58.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:58.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:58.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:58.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:58.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:58.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:58.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:58.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:58.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:58.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:58.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:58.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:58.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:58.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:58.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:58.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:58.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:58.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:58.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:58.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:58.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:58.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:58.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:58.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:58.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:58.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:58.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:58.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:58.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:58.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:59.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:59.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:59.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:59.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:59.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:59.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:59.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:59.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:59.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:59.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:59.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:59.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:59.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:59.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:59.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:59.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:59.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: from='osd.4 [v2:172.21.15.139:6800/217277092,v1:172.21.15.139:6801/217277092]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-24T11:29:59.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-24T11:29:59.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:59.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:59.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:59.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:59.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:59.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:59.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:59.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:59.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:59.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:59.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:59.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:59.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:59.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:59.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:59.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:59.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:59.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:59.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:59.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:59.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:59.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:59.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:59.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:59.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:59.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:59.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:59.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: from='osd.4 [v2:172.21.15.139:6800/217277092,v1:172.21.15.139:6801/217277092]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-24T11:29:59.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-24T11:29:59.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:59.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:59.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:59.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:59.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:59.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:59.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:59.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:59.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:59.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:59.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:59.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:59.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:59.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:59.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:59.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:59.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:59.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: Health check update: Degraded data redundancy: 104/630 objects degraded (16.508%), 24 pgs degraded (PG_DEGRADED) 2024-03-24T11:29:59.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:59.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:59.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:59.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:59.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:59.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:59.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:59.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:59.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:59.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:59.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:59.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:59.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:59.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:59.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:59.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:59.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:59.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:59.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:59.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:59.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:29:59.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:59.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:59.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:59.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:59.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:59.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: Health check update: Degraded data redundancy: 104/630 objects degraded (16.508%), 24 pgs degraded (PG_DEGRADED) 2024-03-24T11:29:59.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:59.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:59.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:59.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:59.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:59.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:59.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:59.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:59.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:59.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:59.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:59.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:59.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:59.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:59.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:29:59.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:29:59.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:29:59.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:29:59.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:29:59.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:29:59 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:00.083 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:00.083 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:00.083 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:00.083 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:00.083 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:00.083 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:00.083 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:00.083 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:00.083 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:00.083 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:00.083 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:00.083 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:00.084 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:00.084 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:00.084 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:00.084 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:00.084 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: from='osd.4 [v2:172.21.15.139:6800/217277092,v1:172.21.15.139:6801/217277092]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-24T11:30:00.084 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-24T11:30:00.084 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:00.084 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:00.084 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:00.084 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:00.084 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:00.084 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:00.084 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:00.084 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:00.084 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:00.085 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:00.085 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:00.085 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:00.085 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:00.085 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:00.085 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:00.085 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:00.085 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:00.086 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: Health check update: Degraded data redundancy: 104/630 objects degraded (16.508%), 24 pgs degraded (PG_DEGRADED) 2024-03-24T11:30:00.086 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:00.086 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:00.086 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:00.086 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:00.086 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:00.086 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:00.086 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:00.086 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:00.086 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:00.086 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:00.086 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:00.086 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:00.087 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:00.087 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:00.087 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:00.087 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:00.087 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:00.087 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:00.087 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:00.087 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:29:59 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:00.358 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4[127378]: 2024-03-24T11:30:00.290+0000 7f09148f0700 -1 osd.4 153 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T11:30:00.947 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:00.947 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:00.948 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: pgmap v242: 161 pgs: 24 active+undersized+degraded, 45 active+undersized, 92 active+clean; 585 KiB data, 212 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 104/630 objects degraded (16.508%) 2024-03-24T11:30:00.948 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:00.948 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:00.948 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:00.948 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:00.948 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:00.948 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:00.948 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:00.948 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:00.948 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:00.948 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-24T11:30:00.948 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: failed to encode map e156 with expected crc 2024-03-24T11:30:00.948 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: osdmap e156: 8 total, 7 up, 8 in 2024-03-24T11:30:00.948 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: from='osd.4 [v2:172.21.15.139:6800/217277092,v1:172.21.15.139:6801/217277092]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:30:00.949 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:30:00.949 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:00.949 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:00.949 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:00.949 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:00.949 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:00.949 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:00.949 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:00.949 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:00.949 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:00.949 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:00.949 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:00.949 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:00.949 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: Health detail: HEALTH_WARN 1 osds down; Degraded data redundancy: 104/630 objects degraded (16.508%), 24 pgs degraded 2024-03-24T11:30:00.949 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: [WRN] OSD_DOWN: 1 osds down 2024-03-24T11:30:00.950 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: osd.4 (root=default,host=smithi139) is down 2024-03-24T11:30:00.950 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: [WRN] PG_DEGRADED: Degraded data redundancy: 104/630 objects degraded (16.508%), 24 pgs degraded 2024-03-24T11:30:00.950 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: pg 2.f is active+undersized+degraded, acting [0,7] 2024-03-24T11:30:00.950 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: pg 3.1 is active+undersized+degraded, acting [0,3] 2024-03-24T11:30:00.950 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: pg 3.3 is active+undersized+degraded, acting [0,6] 2024-03-24T11:30:00.950 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: pg 3.6 is active+undersized+degraded, acting [0,1] 2024-03-24T11:30:00.950 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: pg 3.9 is active+undersized+degraded, acting [2,7] 2024-03-24T11:30:00.950 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: pg 3.a is active+undersized+degraded, acting [6,1] 2024-03-24T11:30:00.950 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: pg 3.b is active+undersized+degraded, acting [3,0] 2024-03-24T11:30:00.950 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: pg 3.e is active+undersized+degraded, acting [7,1] 2024-03-24T11:30:00.950 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: pg 3.f is active+undersized+degraded, acting [7,0] 2024-03-24T11:30:00.950 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: pg 3.11 is active+undersized+degraded, acting [7,6] 2024-03-24T11:30:00.950 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: pg 3.13 is active+undersized+degraded, acting [7,2] 2024-03-24T11:30:00.950 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: pg 3.14 is active+undersized+degraded, acting [7,6] 2024-03-24T11:30:00.951 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: pg 3.15 is active+undersized+degraded, acting [7,3] 2024-03-24T11:30:00.951 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: pg 3.19 is active+undersized+degraded, acting [1,3] 2024-03-24T11:30:00.951 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: pg 3.1a is active+undersized+degraded, acting [1,2] 2024-03-24T11:30:00.951 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: pg 3.1b is active+undersized+degraded, acting [0,7] 2024-03-24T11:30:00.951 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: pg 3.1c is active+undersized+degraded, acting [5,1] 2024-03-24T11:30:00.951 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: pg 3.1d is active+undersized+degraded, acting [5,6] 2024-03-24T11:30:00.951 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: pg 4.9 is active+undersized+degraded, acting [1,3] 2024-03-24T11:30:00.951 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: pg 4.d is active+undersized+degraded, acting [2,1] 2024-03-24T11:30:00.951 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: pg 4.e is active+undersized+degraded, acting [6,1] 2024-03-24T11:30:00.951 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: pg 5.1c is active+undersized+degraded, acting [3,2] 2024-03-24T11:30:00.951 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: pg 5.1d is active+undersized+degraded, acting [1,0] 2024-03-24T11:30:00.951 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: pg 6.1a is active+undersized+degraded, acting [5,1] 2024-03-24T11:30:00.951 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:00.951 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:00.951 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:00.952 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:00.952 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:00.952 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:00.952 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:00.952 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:00.952 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:00.952 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:00.952 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: from='osd.4 ' entity='osd.4' 2024-03-24T11:30:00.952 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:00.952 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:00.952 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:00.952 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:00.952 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:00.952 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:00.953 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:00.953 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:00.953 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:00.953 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:00.953 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:00.953 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:00.953 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:00.953 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:00.953 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:00 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:00.976 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:30:00] "GET /metrics HTTP/1.1" 200 38603 "" "Prometheus/2.33.4" 2024-03-24T11:30:00.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:00.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:00.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: pgmap v242: 161 pgs: 24 active+undersized+degraded, 45 active+undersized, 92 active+clean; 585 KiB data, 212 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 104/630 objects degraded (16.508%) 2024-03-24T11:30:00.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:00.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:00.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:00.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:00.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:00.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:00.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:00.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:00.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:00.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-24T11:30:00.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: failed to encode map e156 with expected crc 2024-03-24T11:30:00.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: osdmap e156: 8 total, 7 up, 8 in 2024-03-24T11:30:00.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: from='osd.4 [v2:172.21.15.139:6800/217277092,v1:172.21.15.139:6801/217277092]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:30:00.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:30:00.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:00.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:00.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:00.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:00.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:00.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:00.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:00.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:00.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:00.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:00.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:00.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:00.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:00.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:00.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: pgmap v242: 161 pgs: 24 active+undersized+degraded, 45 active+undersized, 92 active+clean; 585 KiB data, 212 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 104/630 objects degraded (16.508%) 2024-03-24T11:30:00.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:00.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:00.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:00.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:00.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:00.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:00.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:00.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:00.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:00.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-24T11:30:00.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: failed to encode map e156 with expected crc 2024-03-24T11:30:00.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: osdmap e156: 8 total, 7 up, 8 in 2024-03-24T11:30:00.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: from='osd.4 [v2:172.21.15.139:6800/217277092,v1:172.21.15.139:6801/217277092]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:30:00.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:30:00.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:00.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:00.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:00.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:00.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:00.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:00.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:00.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:00.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:00.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:00.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:00.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:00.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: Health detail: HEALTH_WARN 1 osds down; Degraded data redundancy: 104/630 objects degraded (16.508%), 24 pgs degraded 2024-03-24T11:30:00.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: [WRN] OSD_DOWN: 1 osds down 2024-03-24T11:30:00.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: osd.4 (root=default,host=smithi139) is down 2024-03-24T11:30:00.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: [WRN] PG_DEGRADED: Degraded data redundancy: 104/630 objects degraded (16.508%), 24 pgs degraded 2024-03-24T11:30:00.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: pg 2.f is active+undersized+degraded, acting [0,7] 2024-03-24T11:30:00.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: pg 3.1 is active+undersized+degraded, acting [0,3] 2024-03-24T11:30:00.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: pg 3.3 is active+undersized+degraded, acting [0,6] 2024-03-24T11:30:00.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: pg 3.6 is active+undersized+degraded, acting [0,1] 2024-03-24T11:30:00.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: pg 3.9 is active+undersized+degraded, acting [2,7] 2024-03-24T11:30:00.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: pg 3.a is active+undersized+degraded, acting [6,1] 2024-03-24T11:30:00.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: pg 3.b is active+undersized+degraded, acting [3,0] 2024-03-24T11:30:00.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: pg 3.e is active+undersized+degraded, acting [7,1] 2024-03-24T11:30:00.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: pg 3.f is active+undersized+degraded, acting [7,0] 2024-03-24T11:30:00.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: pg 3.11 is active+undersized+degraded, acting [7,6] 2024-03-24T11:30:00.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: pg 3.13 is active+undersized+degraded, acting [7,2] 2024-03-24T11:30:00.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: pg 3.14 is active+undersized+degraded, acting [7,6] 2024-03-24T11:30:00.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: pg 3.15 is active+undersized+degraded, acting [7,3] 2024-03-24T11:30:00.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: pg 3.19 is active+undersized+degraded, acting [1,3] 2024-03-24T11:30:00.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: pg 3.1a is active+undersized+degraded, acting [1,2] 2024-03-24T11:30:00.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: pg 3.1b is active+undersized+degraded, acting [0,7] 2024-03-24T11:30:00.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: pg 3.1c is active+undersized+degraded, acting [5,1] 2024-03-24T11:30:00.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: pg 3.1d is active+undersized+degraded, acting [5,6] 2024-03-24T11:30:00.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: pg 4.9 is active+undersized+degraded, acting [1,3] 2024-03-24T11:30:00.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: pg 4.d is active+undersized+degraded, acting [2,1] 2024-03-24T11:30:00.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: pg 4.e is active+undersized+degraded, acting [6,1] 2024-03-24T11:30:00.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: pg 5.1c is active+undersized+degraded, acting [3,2] 2024-03-24T11:30:00.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: pg 5.1d is active+undersized+degraded, acting [1,0] 2024-03-24T11:30:00.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: pg 6.1a is active+undersized+degraded, acting [5,1] 2024-03-24T11:30:00.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:00.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:00.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:00.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:00.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:00.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:00.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:00.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:00.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:00.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:00.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: from='osd.4 ' entity='osd.4' 2024-03-24T11:30:00.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:00.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:00.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:00.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:00.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:00.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:00.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:00.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:00.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:00.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:00.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:00.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:00.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:00.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:00.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:00.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: Health detail: HEALTH_WARN 1 osds down; Degraded data redundancy: 104/630 objects degraded (16.508%), 24 pgs degraded 2024-03-24T11:30:00.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: [WRN] OSD_DOWN: 1 osds down 2024-03-24T11:30:00.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: osd.4 (root=default,host=smithi139) is down 2024-03-24T11:30:00.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: [WRN] PG_DEGRADED: Degraded data redundancy: 104/630 objects degraded (16.508%), 24 pgs degraded 2024-03-24T11:30:00.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: pg 2.f is active+undersized+degraded, acting [0,7] 2024-03-24T11:30:00.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: pg 3.1 is active+undersized+degraded, acting [0,3] 2024-03-24T11:30:00.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: pg 3.3 is active+undersized+degraded, acting [0,6] 2024-03-24T11:30:00.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: pg 3.6 is active+undersized+degraded, acting [0,1] 2024-03-24T11:30:00.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: pg 3.9 is active+undersized+degraded, acting [2,7] 2024-03-24T11:30:00.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: pg 3.a is active+undersized+degraded, acting [6,1] 2024-03-24T11:30:00.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: pg 3.b is active+undersized+degraded, acting [3,0] 2024-03-24T11:30:00.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: pg 3.e is active+undersized+degraded, acting [7,1] 2024-03-24T11:30:00.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: pg 3.f is active+undersized+degraded, acting [7,0] 2024-03-24T11:30:00.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: pg 3.11 is active+undersized+degraded, acting [7,6] 2024-03-24T11:30:00.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: pg 3.13 is active+undersized+degraded, acting [7,2] 2024-03-24T11:30:00.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: pg 3.14 is active+undersized+degraded, acting [7,6] 2024-03-24T11:30:00.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: pg 3.15 is active+undersized+degraded, acting [7,3] 2024-03-24T11:30:00.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: pg 3.19 is active+undersized+degraded, acting [1,3] 2024-03-24T11:30:00.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: pg 3.1a is active+undersized+degraded, acting [1,2] 2024-03-24T11:30:00.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: pg 3.1b is active+undersized+degraded, acting [0,7] 2024-03-24T11:30:00.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: pg 3.1c is active+undersized+degraded, acting [5,1] 2024-03-24T11:30:00.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: pg 3.1d is active+undersized+degraded, acting [5,6] 2024-03-24T11:30:00.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: pg 4.9 is active+undersized+degraded, acting [1,3] 2024-03-24T11:30:00.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: pg 4.d is active+undersized+degraded, acting [2,1] 2024-03-24T11:30:00.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: pg 4.e is active+undersized+degraded, acting [6,1] 2024-03-24T11:30:00.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: pg 5.1c is active+undersized+degraded, acting [3,2] 2024-03-24T11:30:00.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: pg 5.1d is active+undersized+degraded, acting [1,0] 2024-03-24T11:30:00.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: pg 6.1a is active+undersized+degraded, acting [5,1] 2024-03-24T11:30:00.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:00.987 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:00.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:00.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:00.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:00.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:00.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:00.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:00.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:00.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:00.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: from='osd.4 ' entity='osd.4' 2024-03-24T11:30:00.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:00.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:00.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:00.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:00.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:00.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:00.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:00.989 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:00.989 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:00.989 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:00.989 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:00.989 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:00.989 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:00.989 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:00.989 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:00 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:01.695 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:01.695 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:01.695 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:01.695 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: failed to encode map e156 with expected crc 2024-03-24T11:30:01.696 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:01.696 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:01.696 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:01.696 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:01.696 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:01.696 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:01.696 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:01.696 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:01.696 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:01.696 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:01.696 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:01.696 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:01.696 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:01.696 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:01.697 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:01.697 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:01.697 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:01.697 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:01.697 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:01.697 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:01.697 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:01.697 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:01.697 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:01.697 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:01.697 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:01.697 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:01.697 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:01.697 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:01.698 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:01.698 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:01.698 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:01.698 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:01.698 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:01.698 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:01.698 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:01.698 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:01.698 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:01.698 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:01.698 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:01.699 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:01.699 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:01.699 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:01.699 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:01.699 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:01.699 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:01.699 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:01.699 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:01.699 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T11:30:01.699 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: osd.4 [v2:172.21.15.139:6800/217277092,v1:172.21.15.139:6801/217277092] boot 2024-03-24T11:30:01.699 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: osdmap e157: 8 total, 8 up, 8 in 2024-03-24T11:30:01.699 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: failed to encode map e157 with expected crc 2024-03-24T11:30:01.699 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T11:30:01.699 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:01.699 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:01.700 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:01.700 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:01.700 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:01.700 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:01.700 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:01.700 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:01.700 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:01.700 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:01.700 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:01.700 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:01.700 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:01.700 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:01.700 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:01.700 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:01.700 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:01.701 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:01.701 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:01.701 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:01 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:01.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:01.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:01.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:01.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: failed to encode map e156 with expected crc 2024-03-24T11:30:01.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:01.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:01.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:01.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:01.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:01.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:01.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:01.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:01.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:01.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:01.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:01.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:01.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:01.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:01.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:01.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:01.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:01.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:01.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:01.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:01.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:01.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:01.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:01.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:01.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:01.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:01.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:01.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:01.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:01.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: failed to encode map e156 with expected crc 2024-03-24T11:30:01.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:01.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:01.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:01.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:01.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:01.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:01.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:01.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:01.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:01.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:01.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:01.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:01.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:01.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:01.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:01.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:01.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:01.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:01.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:01.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:01.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:01.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:01.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:01.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:01.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:01.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:01.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:01.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:01.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:01.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:01.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:01.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:01.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:01.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:01.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:01.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:01.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:01.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:01.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:01.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:01.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:01.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:01.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:01.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:01.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:01.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:01.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:01.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T11:30:01.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: osd.4 [v2:172.21.15.139:6800/217277092,v1:172.21.15.139:6801/217277092] boot 2024-03-24T11:30:01.983 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: osdmap e157: 8 total, 8 up, 8 in 2024-03-24T11:30:01.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: failed to encode map e157 with expected crc 2024-03-24T11:30:01.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T11:30:01.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:01.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:01.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:01.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:01.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:01.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:01.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:01.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:01.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:01.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:01.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:01.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:01.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:01.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:01.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:01.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:01.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:01.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:01.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:01.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:01.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:01.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:01.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:01.985 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:01.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:01.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:01.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:01.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:01.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:01.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:01.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:01.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:01.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:01.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:01.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:01.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:01.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:01.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:01.986 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:01.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:01.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:01.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T11:30:01.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: osd.4 [v2:172.21.15.139:6800/217277092,v1:172.21.15.139:6801/217277092] boot 2024-03-24T11:30:01.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: osdmap e157: 8 total, 8 up, 8 in 2024-03-24T11:30:01.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: failed to encode map e157 with expected crc 2024-03-24T11:30:01.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T11:30:01.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:01.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:01.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:01.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:01.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:01.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:01.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:01.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:01.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:01.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:01.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:01.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:01.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:01.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:01.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:01.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:01.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:01.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:01.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:01.988 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:01 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:02.727 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:02.727 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:02.727 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:02.727 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: failed to encode map e157 with expected crc 2024-03-24T11:30:02.727 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: failed to encode map e157 with expected crc 2024-03-24T11:30:02.727 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:02.727 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: pgmap v245: 161 pgs: 3 peering, 24 active+undersized+degraded, 42 active+undersized, 92 active+clean; 585 KiB data, 212 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 104/630 objects degraded (16.508%) 2024-03-24T11:30:02.727 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:02.727 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:02.727 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:02.728 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:02.728 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:02.728 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:02.728 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:02.728 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:02.728 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:02.728 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:02.728 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:02.728 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:02.728 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:02.728 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:02.728 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:02.728 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:02.728 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:02.728 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:02.728 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:02.729 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:02.729 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:02.729 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:02.729 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:02.729 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:02.729 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:02.729 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:02.729 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:02.729 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:02.729 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:02.729 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:02.729 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:02.729 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: failed to encode map e158 with expected crc 2024-03-24T11:30:02.729 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: failed to encode map e158 with expected crc 2024-03-24T11:30:02.729 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: osdmap e158: 8 total, 8 up, 8 in 2024-03-24T11:30:02.729 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:02.729 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:02.730 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:02.730 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:02.730 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:02.730 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:02.730 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:02.730 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:02.730 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:02.730 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:02.730 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:02.730 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:02.730 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:02.730 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:02.730 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:02.958 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:30:02.851+0000 7f4e4aad7700 -1 mgr.server reply reply (16) Device or resource busy unsafe to stop osd(s) at this time (14 PGs are or would become offline) 2024-03-24T11:30:02.958 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:02.958 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:02.958 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:02.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: failed to encode map e157 with expected crc 2024-03-24T11:30:02.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: failed to encode map e157 with expected crc 2024-03-24T11:30:02.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:02.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: pgmap v245: 161 pgs: 3 peering, 24 active+undersized+degraded, 42 active+undersized, 92 active+clean; 585 KiB data, 212 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 104/630 objects degraded (16.508%) 2024-03-24T11:30:02.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:02.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:02.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:02.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:02.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:02.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:02.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:02.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:02.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:02.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:02.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:02.960 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:02.960 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:02.960 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:02.960 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:02.960 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:02.960 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:02.960 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:02.960 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:02.960 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:02.960 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:02.960 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:02.960 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:02.961 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:02.961 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:02.961 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:02.961 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: failed to encode map e157 with expected crc 2024-03-24T11:30:02.961 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: failed to encode map e157 with expected crc 2024-03-24T11:30:02.961 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:02.961 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: pgmap v245: 161 pgs: 3 peering, 24 active+undersized+degraded, 42 active+undersized, 92 active+clean; 585 KiB data, 212 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 104/630 objects degraded (16.508%) 2024-03-24T11:30:02.961 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:02.961 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:02.961 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:02.961 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:02.961 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:02.962 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:02.962 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:02.962 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:02.962 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:02.962 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:02.962 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:02.962 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:02.962 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:02.962 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:02.962 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:02.962 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:02.962 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:02.962 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:02.962 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:02.962 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:02.963 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:02.963 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:02.963 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:02.963 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:02.963 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:02.963 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:02.963 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:02.963 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:02.963 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:02.963 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:02.964 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:02.964 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: failed to encode map e158 with expected crc 2024-03-24T11:30:02.964 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: failed to encode map e158 with expected crc 2024-03-24T11:30:02.964 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: osdmap e158: 8 total, 8 up, 8 in 2024-03-24T11:30:02.964 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:02.964 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:02.964 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:02.964 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:02.964 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:02.964 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:02.964 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:02.964 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:02.964 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:02.964 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:02.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:02.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:02.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:02.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:02.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:02.965 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:02.965 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:02.965 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:02.965 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:02.965 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:02.965 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:02.965 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:02.966 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:02.966 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: failed to encode map e158 with expected crc 2024-03-24T11:30:02.966 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: failed to encode map e158 with expected crc 2024-03-24T11:30:02.966 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: osdmap e158: 8 total, 8 up, 8 in 2024-03-24T11:30:02.966 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:02.966 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:02.966 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:02.966 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:02.966 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:02.966 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:02.966 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:02.966 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:02.966 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:02.966 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:02.967 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:02.967 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:02.967 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:02.967 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:02.967 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:03.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:03 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:03.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:03 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:03.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:03 smithi139 ceph-mon[91894]: failed to encode map e158 with expected crc 2024-03-24T11:30:03.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:03 smithi139 ceph-mon[91894]: failed to encode map e158 with expected crc 2024-03-24T11:30:03.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:03 smithi139 ceph-mon[91894]: failed to encode map e158 with expected crc 2024-03-24T11:30:03.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:03 smithi139 ceph-mon[91894]: failed to encode map e158 with expected crc 2024-03-24T11:30:03.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:03 smithi139 ceph-mon[91894]: failed to encode map e158 with expected crc 2024-03-24T11:30:03.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:03 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:03.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:03 smithi139 ceph-mon[91894]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:30:03.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:03 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:03.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:03 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:03.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:03.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:03.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:03.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:03.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:03.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:03 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:03.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:03.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:03.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:03.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:03.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:03.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:03.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:03.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-24T11:30:03.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:03 smithi139 ceph-mon[91894]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-24T11:30:03.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:03 smithi139 ceph-mon[91894]: Upgrade: unsafe to stop osd(s) at this time (14 PGs are or would become offline) 2024-03-24T11:30:03.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:03.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:03.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[90553]: failed to encode map e158 with expected crc 2024-03-24T11:30:03.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[90553]: failed to encode map e158 with expected crc 2024-03-24T11:30:03.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[90553]: failed to encode map e158 with expected crc 2024-03-24T11:30:03.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[90553]: failed to encode map e158 with expected crc 2024-03-24T11:30:03.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[90553]: failed to encode map e158 with expected crc 2024-03-24T11:30:03.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:03.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:30:03.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:03.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:03.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:03.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:03.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:03.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:03.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:03.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:03.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:03.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:03.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:03.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:03.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:03.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:03.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:03.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-24T11:30:03.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[90553]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-24T11:30:03.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[90553]: Upgrade: unsafe to stop osd(s) at this time (14 PGs are or would become offline) 2024-03-24T11:30:03.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:03.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:03.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[98643]: failed to encode map e158 with expected crc 2024-03-24T11:30:03.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[98643]: failed to encode map e158 with expected crc 2024-03-24T11:30:03.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[98643]: failed to encode map e158 with expected crc 2024-03-24T11:30:03.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[98643]: failed to encode map e158 with expected crc 2024-03-24T11:30:03.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[98643]: failed to encode map e158 with expected crc 2024-03-24T11:30:03.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:03.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[98643]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:30:03.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:03.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:03.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:03.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:03.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:03.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:03.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:03.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:03.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:03.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:03.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:03.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:03.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:03.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:03.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:03.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-24T11:30:03.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[98643]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-24T11:30:03.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:03 smithi062 ceph-mon[98643]: Upgrade: unsafe to stop osd(s) at this time (14 PGs are or would become offline) 2024-03-24T11:30:05.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:04 smithi139 ceph-mon[91894]: pgmap v247: 161 pgs: 28 peering, 14 active+undersized+degraded, 20 active+undersized, 99 active+clean; 585 KiB data, 212 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 48/630 objects degraded (7.619%) 2024-03-24T11:30:05.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:04 smithi139 ceph-mon[91894]: Health check update: Degraded data redundancy: 48/630 objects degraded (7.619%), 14 pgs degraded (PG_DEGRADED) 2024-03-24T11:30:05.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:04 smithi062 ceph-mon[98643]: pgmap v247: 161 pgs: 28 peering, 14 active+undersized+degraded, 20 active+undersized, 99 active+clean; 585 KiB data, 212 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 48/630 objects degraded (7.619%) 2024-03-24T11:30:05.271 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:04 smithi062 ceph-mon[98643]: Health check update: Degraded data redundancy: 48/630 objects degraded (7.619%), 14 pgs degraded (PG_DEGRADED) 2024-03-24T11:30:05.271 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:04 smithi062 ceph-mon[90553]: pgmap v247: 161 pgs: 28 peering, 14 active+undersized+degraded, 20 active+undersized, 99 active+clean; 585 KiB data, 212 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 48/630 objects degraded (7.619%) 2024-03-24T11:30:05.271 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:04 smithi062 ceph-mon[90553]: Health check update: Degraded data redundancy: 48/630 objects degraded (7.619%), 14 pgs degraded (PG_DEGRADED) 2024-03-24T11:30:06.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:05 smithi062 ceph-mon[90553]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 48/630 objects degraded (7.619%), 14 pgs degraded) 2024-03-24T11:30:06.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:05 smithi062 ceph-mon[90553]: Cluster is now healthy 2024-03-24T11:30:06.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:05 smithi062 ceph-mon[98643]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 48/630 objects degraded (7.619%), 14 pgs degraded) 2024-03-24T11:30:06.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:05 smithi062 ceph-mon[98643]: Cluster is now healthy 2024-03-24T11:30:06.256 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:05 smithi139 ceph-mon[91894]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 48/630 objects degraded (7.619%), 14 pgs degraded) 2024-03-24T11:30:06.256 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:05 smithi139 ceph-mon[91894]: Cluster is now healthy 2024-03-24T11:30:07.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:06 smithi062 ceph-mon[90553]: pgmap v248: 161 pgs: 28 peering, 133 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:30:07.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:06 smithi062 ceph-mon[98643]: pgmap v248: 161 pgs: 28 peering, 133 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:30:07.295 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:06 smithi139 ceph-mon[91894]: pgmap v248: 161 pgs: 28 peering, 133 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:30:09.060 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:08 smithi139 ceph-mon[91894]: pgmap v249: 161 pgs: 161 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 671 B/s rd, 0 op/s 2024-03-24T11:30:09.060 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:09.060 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:09.061 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:09.061 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:09.061 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:09.061 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:09.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:08 smithi062 ceph-mon[98643]: pgmap v249: 161 pgs: 161 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 671 B/s rd, 0 op/s 2024-03-24T11:30:09.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:09.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:09.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:09.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:09.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:09.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:09.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:08 smithi062 ceph-mon[90553]: pgmap v249: 161 pgs: 161 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 671 B/s rd, 0 op/s 2024-03-24T11:30:09.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:09.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:09.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:09.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:09.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:09.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:10.356 INFO:teuthology.orchestra.run.smithi062.stdout:true 2024-03-24T11:30:11.025 INFO:teuthology.orchestra.run.smithi062.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T11:30:11.025 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi062 smithi062 running 1s ago 19m - - 2024-03-24T11:30:11.025 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi139 smithi139 running 2s ago 19m - - 2024-03-24T11:30:11.025 INFO:teuthology.orchestra.run.smithi062.stdout:alertmanager.a smithi062 *:9093,9094 running (9m) 1s ago 14m 25.5M - 0.23.0 ba2b418f427c 818298d58140 2024-03-24T11:30:11.025 INFO:teuthology.orchestra.run.smithi062.stdout:grafana.a smithi139 *:3000 running (8m) 2s ago 13m 54.0M - 8.3.5 dad864ee21e9 b48167e7171e 2024-03-24T11:30:11.025 INFO:teuthology.orchestra.run.smithi062.stdout:iscsi.foo.smithi062.dggsvk smithi062 running (7m) 1s ago 12m 66.2M - 3.5 e1d6a67b021e 0a98806ec839 2024-03-24T11:30:11.025 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.x smithi139 *:8443,9283,8765 running (6m) 2s ago 18m 452M - 19.0.0-1667-gdb0330b1 4f325f8440e9 d94987b11387 2024-03-24T11:30:11.025 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.y smithi062 *:8443,9283,8765 running (8m) 1s ago 20m 532M - 19.0.0-1667-gdb0330b1 4f325f8440e9 ca1a49028d65 2024-03-24T11:30:11.026 INFO:teuthology.orchestra.run.smithi062.stdout:mon.a smithi062 running (6m) 1s ago 20m 72.8M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 f7afe6e28c88 2024-03-24T11:30:11.026 INFO:teuthology.orchestra.run.smithi062.stdout:mon.b smithi139 running (5m) 2s ago 19m 50.0M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 ca357beea4dd 2024-03-24T11:30:11.026 INFO:teuthology.orchestra.run.smithi062.stdout:mon.c smithi062 running (5m) 1s ago 19m 59.0M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 459e7449ec39 2024-03-24T11:30:11.026 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.a smithi062 *:9100 running (9m) 1s ago 15m 15.8M - 1.3.1 1dbe0e931976 79d5a02a999f 2024-03-24T11:30:11.026 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.b smithi139 *:9100 running (9m) 2s ago 14m 17.2M - 1.3.1 1dbe0e931976 8f9f4608e465 2024-03-24T11:30:11.026 INFO:teuthology.orchestra.run.smithi062.stdout:osd.0 smithi062 running (4m) 1s ago 18m 59.7M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 dfe4920b49c4 2024-03-24T11:30:11.026 INFO:teuthology.orchestra.run.smithi062.stdout:osd.1 smithi062 running (3m) 1s ago 18m 51.7M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 c443b3fe97c1 2024-03-24T11:30:11.026 INFO:teuthology.orchestra.run.smithi062.stdout:osd.2 smithi062 running (2m) 1s ago 17m 46.4M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 0eba40dc96b3 2024-03-24T11:30:11.026 INFO:teuthology.orchestra.run.smithi062.stdout:osd.3 smithi062 running (72s) 1s ago 17m 54.7M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 7501d2e4c718 2024-03-24T11:30:11.026 INFO:teuthology.orchestra.run.smithi062.stdout:osd.4 smithi139 running (15s) 2s ago 17m 49.7M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 5d0f7f05ec6d 2024-03-24T11:30:11.026 INFO:teuthology.orchestra.run.smithi062.stdout:osd.5 smithi139 running (16m) 2s ago 16m 65.1M 4096M 17.2.0 e1d6a67b021e dd0abd599a14 2024-03-24T11:30:11.027 INFO:teuthology.orchestra.run.smithi062.stdout:osd.6 smithi139 running (16m) 2s ago 16m 64.4M 4096M 17.2.0 e1d6a67b021e a4b1289aca69 2024-03-24T11:30:11.027 INFO:teuthology.orchestra.run.smithi062.stdout:osd.7 smithi139 running (15m) 2s ago 15m 66.8M 4096M 17.2.0 e1d6a67b021e bc7fadbc1384 2024-03-24T11:30:11.027 INFO:teuthology.orchestra.run.smithi062.stdout:prometheus.a smithi139 *:9095 running (7m) 2s ago 15m 71.3M - 2.33.4 514e6a882f6e 2ed47f810d33 2024-03-24T11:30:11.027 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi062.sacpds smithi062 *:8000 running (13m) 1s ago 13m 75.0M - 17.2.0 e1d6a67b021e 89f4ac308e98 2024-03-24T11:30:11.027 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi139.yljesh smithi139 *:8000 running (13m) 2s ago 13m 74.1M - 17.2.0 e1d6a67b021e e189bdc12cae 2024-03-24T11:30:11.027 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi062.xvpxnk smithi062 *:80 running (13m) 1s ago 13m 74.8M - 17.2.0 e1d6a67b021e d1aa74adab44 2024-03-24T11:30:11.027 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi139.pbtrmp smithi139 *:80 running (13m) 2s ago 13m 77.9M - 17.2.0 e1d6a67b021e 7f8e336755d7 2024-03-24T11:30:11.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:30:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:30:10] "GET /metrics HTTP/1.1" 200 38603 "" "Prometheus/2.33.4" 2024-03-24T11:30:11.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:10 smithi062 ceph-mon[90553]: pgmap v250: 161 pgs: 161 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:30:11.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:11.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:11.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:11.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:10 smithi062 ceph-mon[98643]: pgmap v250: 161 pgs: 161 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:30:11.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:11.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:11.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:11.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:10 smithi139 ceph-mon[91894]: pgmap v250: 161 pgs: 161 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:30:11.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:11.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:11.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:11.433 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:30:11.433 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": { 2024-03-24T11:30:11.433 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 3 2024-03-24T11:30:11.434 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:30:11.434 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": { 2024-03-24T11:30:11.434 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-24T11:30:11.434 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:30:11.434 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": { 2024-03-24T11:30:11.434 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 3, 2024-03-24T11:30:11.434 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 5 2024-03-24T11:30:11.434 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:30:11.434 INFO:teuthology.orchestra.run.smithi062.stdout: "rgw": { 2024-03-24T11:30:11.434 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4 2024-03-24T11:30:11.434 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:30:11.434 INFO:teuthology.orchestra.run.smithi062.stdout: "overall": { 2024-03-24T11:30:11.434 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 7, 2024-03-24T11:30:11.434 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 10 2024-03-24T11:30:11.434 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-24T11:30:11.435 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:30:11.773 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:30:11.774 INFO:teuthology.orchestra.run.smithi062.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69", 2024-03-24T11:30:11.774 INFO:teuthology.orchestra.run.smithi062.stdout: "in_progress": true, 2024-03-24T11:30:11.774 INFO:teuthology.orchestra.run.smithi062.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T11:30:11.774 INFO:teuthology.orchestra.run.smithi062.stdout: "services_complete": [ 2024-03-24T11:30:11.774 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr", 2024-03-24T11:30:11.774 INFO:teuthology.orchestra.run.smithi062.stdout: "mon" 2024-03-24T11:30:11.774 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-03-24T11:30:11.774 INFO:teuthology.orchestra.run.smithi062.stdout: "progress": "10/23 daemons upgraded", 2024-03-24T11:30:11.774 INFO:teuthology.orchestra.run.smithi062.stdout: "message": "Currently upgrading osd daemons", 2024-03-24T11:30:11.774 INFO:teuthology.orchestra.run.smithi062.stdout: "is_paused": false 2024-03-24T11:30:11.774 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:30:12.178 INFO:teuthology.orchestra.run.smithi062.stdout:HEALTH_OK 2024-03-24T11:30:12.179 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:11 smithi062 ceph-mon[90553]: from='client.44550 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:30:12.179 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:11 smithi062 ceph-mon[90553]: from='client.44556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:30:12.179 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:11 smithi062 ceph-mon[90553]: from='client.64531 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:30:12.179 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:12.179 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:30:12.179 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:11 smithi062 ceph-mon[90553]: from='client.? 172.21.15.62:0/1141278220' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:12.179 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:11 smithi062 ceph-mon[98643]: from='client.44550 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:30:12.179 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:11 smithi062 ceph-mon[98643]: from='client.44556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:30:12.179 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:11 smithi062 ceph-mon[98643]: from='client.64531 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:30:12.179 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:12.180 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:30:12.180 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:11 smithi062 ceph-mon[98643]: from='client.? 172.21.15.62:0/1141278220' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:12.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:11 smithi139 ceph-mon[91894]: from='client.44550 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:30:12.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:11 smithi139 ceph-mon[91894]: from='client.44556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:30:12.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:11 smithi139 ceph-mon[91894]: from='client.64531 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:30:12.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:12.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:30:12.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:11 smithi139 ceph-mon[91894]: from='client.? 172.21.15.62:0/1141278220' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:13.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:12 smithi062 ceph-mon[90553]: pgmap v251: 161 pgs: 161 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T11:30:13.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:12 smithi062 ceph-mon[90553]: from='client.44574 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:30:13.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:12 smithi062 ceph-mon[90553]: from='client.? 172.21.15.62:0/410355999' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:30:13.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:12 smithi062 ceph-mon[98643]: pgmap v251: 161 pgs: 161 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T11:30:13.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:12 smithi062 ceph-mon[98643]: from='client.44574 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:30:13.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:12 smithi062 ceph-mon[98643]: from='client.? 172.21.15.62:0/410355999' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:30:13.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:12 smithi139 ceph-mon[91894]: pgmap v251: 161 pgs: 161 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T11:30:13.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:12 smithi139 ceph-mon[91894]: from='client.44574 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:30:13.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:12 smithi139 ceph-mon[91894]: from='client.? 172.21.15.62:0/410355999' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:30:14.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:13 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:30:14.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:13 smithi062 ceph-mon[98643]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:30:14.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:13 smithi139 ceph-mon[91894]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:30:15.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:14 smithi062 ceph-mon[90553]: pgmap v252: 161 pgs: 161 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 920 B/s rd, 0 op/s 2024-03-24T11:30:15.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:14 smithi062 ceph-mon[98643]: pgmap v252: 161 pgs: 161 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 920 B/s rd, 0 op/s 2024-03-24T11:30:15.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:14 smithi139 ceph-mon[91894]: pgmap v252: 161 pgs: 161 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 920 B/s rd, 0 op/s 2024-03-24T11:30:17.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:16 smithi062 ceph-mon[98643]: pgmap v253: 161 pgs: 161 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:30:17.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:17.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:16 smithi062 ceph-mon[90553]: pgmap v253: 161 pgs: 161 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:30:17.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:17.235 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:16 smithi139 ceph-mon[91894]: pgmap v253: 161 pgs: 161 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:30:17.236 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:18.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-24T11:30:18.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-24T11:30:18.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-24T11:30:19.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:18 smithi139 ceph-mon[91894]: pgmap v254: 161 pgs: 161 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:30:19.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:18 smithi139 ceph-mon[91894]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-24T11:30:19.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:18 smithi139 ceph-mon[91894]: Upgrade: osd.5 is safe to restart 2024-03-24T11:30:19.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:18 smithi062 ceph-mon[90553]: pgmap v254: 161 pgs: 161 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:30:19.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:18 smithi062 ceph-mon[90553]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-24T11:30:19.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:18 smithi062 ceph-mon[90553]: Upgrade: osd.5 is safe to restart 2024-03-24T11:30:19.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:18 smithi062 ceph-mon[98643]: pgmap v254: 161 pgs: 161 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:30:19.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:18 smithi062 ceph-mon[98643]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-24T11:30:19.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:18 smithi062 ceph-mon[98643]: Upgrade: osd.5 is safe to restart 2024-03-24T11:30:20.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:20 smithi139 ceph-mon[91894]: pgmap v255: 161 pgs: 161 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:30:20.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:20 smithi139 ceph-mon[91894]: Upgrade: Updating osd.5 2024-03-24T11:30:20.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:20.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-24T11:30:20.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:20.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:20 smithi139 ceph-mon[91894]: Deploying daemon osd.5 on smithi139 2024-03-24T11:30:20.977 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:30:20 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:30:20] "GET /metrics HTTP/1.1" 200 38593 "" "Prometheus/2.33.4" 2024-03-24T11:30:20.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:20 smithi062 ceph-mon[90553]: pgmap v255: 161 pgs: 161 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:30:20.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:20 smithi062 ceph-mon[90553]: Upgrade: Updating osd.5 2024-03-24T11:30:20.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:20.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-24T11:30:20.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:20.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:20 smithi062 ceph-mon[90553]: Deploying daemon osd.5 on smithi139 2024-03-24T11:30:20.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:20 smithi062 ceph-mon[98643]: pgmap v255: 161 pgs: 161 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:30:20.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:20 smithi062 ceph-mon[98643]: Upgrade: Updating osd.5 2024-03-24T11:30:20.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:20.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-24T11:30:20.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:20.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:20 smithi062 ceph-mon[98643]: Deploying daemon osd.5 on smithi139 2024-03-24T11:30:22.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:22 smithi062 ceph-mon[90553]: pgmap v256: 161 pgs: 161 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:30:22.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:22 smithi062 ceph-mon[98643]: pgmap v256: 161 pgs: 161 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:30:22.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:22 smithi139 ceph-mon[91894]: pgmap v256: 161 pgs: 161 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:30:23.268 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:30:22 smithi139 systemd[1]: Stopping Ceph osd.5 for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:30:23.590 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:30:23 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5[33215]: 2024-03-24T11:30:23.328+0000 7f7edc86a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T11:30:23.590 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:30:23 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5[33215]: 2024-03-24T11:30:23.328+0000 7f7edc86a700 -1 osd.5 158 *** Got signal Terminated *** 2024-03-24T11:30:23.590 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:30:23 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5[33215]: 2024-03-24T11:30:23.328+0000 7f7edc86a700 -1 osd.5 158 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T11:30:23.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:23 smithi139 ceph-mon[91894]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:30:23.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:23 smithi139 ceph-mon[91894]: osd.5 marked itself down and dead 2024-03-24T11:30:23.915 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:23 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:30:23.915 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:23 smithi062 ceph-mon[90553]: osd.5 marked itself down and dead 2024-03-24T11:30:23.915 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:23 smithi062 ceph-mon[98643]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:30:23.915 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:23 smithi062 ceph-mon[98643]: osd.5 marked itself down and dead 2024-03-24T11:30:24.358 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:30:24 smithi139 podman[131512]: 2024-03-24 11:30:24.091536563 +0000 UTC m=+0.899496517 container died dd0abd599a142cd5ee1baf20193c01e79dce16fea5dccfcc017be4c94a8c7cb5 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, vendor=Red Hat, Inc., distribution-scope=public, io.k8s.display-name=CentOS Stream 8, GIT_REPO=https://github.com/ceph/ceph-container.git, architecture=x86_64, CEPH_POINT_RELEASE=-17.2.0, vcs-type=git, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, ceph=True, io.buildah.version=1.19.8, maintainer=Guillaume Abrioux , io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.openshift.expose-services=, build-date=2022-05-03T08:36:31.336870, com.redhat.component=centos-stream-container, RELEASE=HEAD, GIT_BRANCH=HEAD, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, name=centos-stream, io.openshift.tags=base centos centos-stream, version=8, release=754, GIT_CLEAN=True, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb) 2024-03-24T11:30:24.713 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:30:24 smithi139 podman[131512]: 2024-03-24 11:30:24.528861315 +0000 UTC m=+1.336821268 container cleanup dd0abd599a142cd5ee1baf20193c01e79dce16fea5dccfcc017be4c94a8c7cb5 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, version=8, GIT_REPO=https://github.com/ceph/ceph-container.git, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., GIT_BRANCH=HEAD, architecture=x86_64, distribution-scope=public, build-date=2022-05-03T08:36:31.336870, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, io.k8s.display-name=CentOS Stream 8, GIT_CLEAN=True, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, io.buildah.version=1.19.8, name=centos-stream, com.redhat.component=centos-stream-container, maintainer=Guillaume Abrioux , RELEASE=HEAD, io.openshift.expose-services=, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, io.openshift.tags=base centos centos-stream, release=754, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, CEPH_POINT_RELEASE=-17.2.0, ceph=True, vendor=Red Hat, Inc., GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, vcs-type=git) 2024-03-24T11:30:24.714 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:30:24 smithi139 bash[131512]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5 2024-03-24T11:30:24.714 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:24 smithi139 ceph-mon[91894]: pgmap v257: 161 pgs: 161 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:30:24.714 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:24 smithi139 ceph-mon[91894]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T11:30:24.714 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:24 smithi139 ceph-mon[91894]: failed to encode map e159 with expected crc 2024-03-24T11:30:24.714 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:24 smithi139 ceph-mon[91894]: failed to encode map e159 with expected crc 2024-03-24T11:30:24.714 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:24 smithi139 ceph-mon[91894]: failed to encode map e159 with expected crc 2024-03-24T11:30:24.714 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:24 smithi139 ceph-mon[91894]: failed to encode map e159 with expected crc 2024-03-24T11:30:24.714 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:24 smithi139 ceph-mon[91894]: osdmap e159: 8 total, 7 up, 8 in 2024-03-24T11:30:24.715 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:24 smithi139 ceph-mon[91894]: failed to encode map e159 with expected crc 2024-03-24T11:30:24.715 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:24 smithi139 ceph-mon[91894]: failed to encode map e159 with expected crc 2024-03-24T11:30:24.715 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:24 smithi139 ceph-mon[91894]: failed to encode map e159 with expected crc 2024-03-24T11:30:24.715 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:24 smithi139 ceph-mon[91894]: failed to encode map e159 with expected crc 2024-03-24T11:30:24.959 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:24 smithi062 ceph-mon[98643]: pgmap v257: 161 pgs: 161 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:30:24.959 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:24 smithi062 ceph-mon[98643]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T11:30:24.959 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:24 smithi062 ceph-mon[98643]: failed to encode map e159 with expected crc 2024-03-24T11:30:24.959 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:24 smithi062 ceph-mon[98643]: failed to encode map e159 with expected crc 2024-03-24T11:30:24.959 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:24 smithi062 ceph-mon[98643]: failed to encode map e159 with expected crc 2024-03-24T11:30:24.959 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:24 smithi062 ceph-mon[98643]: failed to encode map e159 with expected crc 2024-03-24T11:30:24.959 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:24 smithi062 ceph-mon[98643]: osdmap e159: 8 total, 7 up, 8 in 2024-03-24T11:30:24.959 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:24 smithi062 ceph-mon[98643]: failed to encode map e159 with expected crc 2024-03-24T11:30:24.959 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:24 smithi062 ceph-mon[98643]: failed to encode map e159 with expected crc 2024-03-24T11:30:24.959 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:24 smithi062 ceph-mon[98643]: failed to encode map e159 with expected crc 2024-03-24T11:30:24.959 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:24 smithi062 ceph-mon[98643]: failed to encode map e159 with expected crc 2024-03-24T11:30:24.959 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:24 smithi062 ceph-mon[90553]: pgmap v257: 161 pgs: 161 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:30:24.960 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:24 smithi062 ceph-mon[90553]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T11:30:24.960 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:24 smithi062 ceph-mon[90553]: failed to encode map e159 with expected crc 2024-03-24T11:30:24.960 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:24 smithi062 ceph-mon[90553]: failed to encode map e159 with expected crc 2024-03-24T11:30:24.960 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:24 smithi062 ceph-mon[90553]: failed to encode map e159 with expected crc 2024-03-24T11:30:24.960 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:24 smithi062 ceph-mon[90553]: failed to encode map e159 with expected crc 2024-03-24T11:30:24.960 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:24 smithi062 ceph-mon[90553]: osdmap e159: 8 total, 7 up, 8 in 2024-03-24T11:30:24.960 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:24 smithi062 ceph-mon[90553]: failed to encode map e159 with expected crc 2024-03-24T11:30:24.960 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:24 smithi062 ceph-mon[90553]: failed to encode map e159 with expected crc 2024-03-24T11:30:24.960 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:24 smithi062 ceph-mon[90553]: failed to encode map e159 with expected crc 2024-03-24T11:30:24.960 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:24 smithi062 ceph-mon[90553]: failed to encode map e159 with expected crc 2024-03-24T11:30:24.965 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:30:24 smithi139 podman[131539]: 2024-03-24 11:30:24.713677979 +0000 UTC m=+0.625440305 container remove dd0abd599a142cd5ee1baf20193c01e79dce16fea5dccfcc017be4c94a8c7cb5 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5, name=centos-stream, io.openshift.tags=base centos centos-stream, GIT_REPO=https://github.com/ceph/ceph-container.git, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, release=754, architecture=x86_64, distribution-scope=public, RELEASE=HEAD, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=-17.2.0, build-date=2022-05-03T08:36:31.336870, ceph=True, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, vcs-type=git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , version=8, io.buildah.version=1.19.8, vendor=Red Hat, Inc., io.k8s.display-name=CentOS Stream 8, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, com.redhat.component=centos-stream-container, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., com.redhat.license_terms=https://centos.org/legal/licensing-policy/, io.openshift.expose-services=) 2024-03-24T11:30:25.222 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:30:25 smithi139 podman[131738]: 2024-03-24 11:30:25.062812164 +0000 UTC m=+0.019546700 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:30:25.542 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:30:25 smithi139 podman[131738]: 2024-03-24 11:30:25.223131202 +0000 UTC m=+0.179865733 container create 5d991a200fbc8948c6ef7f76840d7facff8373204656233cc6e694045be7f3f0 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5-deactivate, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307) 2024-03-24T11:30:25.542 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:30:25 smithi139 podman[131738]: 2024-03-24 11:30:25.458973049 +0000 UTC m=+0.415707580 container init 5d991a200fbc8948c6ef7f76840d7facff8373204656233cc6e694045be7f3f0 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-db0330b, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-24T11:30:25.542 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:30:25 smithi139 podman[131738]: 2024-03-24 11:30:25.462109509 +0000 UTC m=+0.418844046 container start 5d991a200fbc8948c6ef7f76840d7facff8373204656233cc6e694045be7f3f0 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-db0330b, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-24T11:30:25.542 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:30:25 smithi139 podman[131738]: 2024-03-24 11:30:25.54211508 +0000 UTC m=+0.498849605 container attach 5d991a200fbc8948c6ef7f76840d7facff8373204656233cc6e694045be7f3f0 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5-deactivate, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-24T11:30:25.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:25 smithi062 ceph-mon[90553]: failed to encode map e160 with expected crc 2024-03-24T11:30:25.743 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:25 smithi062 ceph-mon[90553]: failed to encode map e160 with expected crc 2024-03-24T11:30:25.743 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:25 smithi062 ceph-mon[90553]: failed to encode map e160 with expected crc 2024-03-24T11:30:25.743 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:25 smithi062 ceph-mon[90553]: failed to encode map e160 with expected crc 2024-03-24T11:30:25.743 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:25 smithi062 ceph-mon[90553]: osdmap e160: 8 total, 7 up, 8 in 2024-03-24T11:30:25.743 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:25 smithi062 ceph-mon[90553]: failed to encode map e160 with expected crc 2024-03-24T11:30:25.744 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:25 smithi062 ceph-mon[90553]: failed to encode map e160 with expected crc 2024-03-24T11:30:25.744 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:25 smithi062 ceph-mon[98643]: failed to encode map e160 with expected crc 2024-03-24T11:30:25.744 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:25 smithi062 ceph-mon[98643]: failed to encode map e160 with expected crc 2024-03-24T11:30:25.744 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:25 smithi062 ceph-mon[98643]: failed to encode map e160 with expected crc 2024-03-24T11:30:25.744 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:25 smithi062 ceph-mon[98643]: failed to encode map e160 with expected crc 2024-03-24T11:30:25.744 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:25 smithi062 ceph-mon[98643]: osdmap e160: 8 total, 7 up, 8 in 2024-03-24T11:30:25.744 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:25 smithi062 ceph-mon[98643]: failed to encode map e160 with expected crc 2024-03-24T11:30:25.744 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:25 smithi062 ceph-mon[98643]: failed to encode map e160 with expected crc 2024-03-24T11:30:25.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:25 smithi139 ceph-mon[91894]: failed to encode map e160 with expected crc 2024-03-24T11:30:25.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:25 smithi139 ceph-mon[91894]: failed to encode map e160 with expected crc 2024-03-24T11:30:25.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:25 smithi139 ceph-mon[91894]: failed to encode map e160 with expected crc 2024-03-24T11:30:25.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:25 smithi139 ceph-mon[91894]: failed to encode map e160 with expected crc 2024-03-24T11:30:25.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:25 smithi139 ceph-mon[91894]: osdmap e160: 8 total, 7 up, 8 in 2024-03-24T11:30:25.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:25 smithi139 ceph-mon[91894]: failed to encode map e160 with expected crc 2024-03-24T11:30:25.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:25 smithi139 ceph-mon[91894]: failed to encode map e160 with expected crc 2024-03-24T11:30:26.313 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:30:25 smithi139 podman[131738]: 2024-03-24 11:30:25.957216663 +0000 UTC m=+0.913951206 container died 5d991a200fbc8948c6ef7f76840d7facff8373204656233cc6e694045be7f3f0 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=squid-db0330b, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T11:30:26.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:26 smithi062 ceph-mon[98643]: pgmap v260: 161 pgs: 3 active+undersized+degraded, 17 peering, 11 stale+active+clean, 6 active+undersized, 124 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 14/630 objects degraded (2.222%) 2024-03-24T11:30:26.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:26 smithi062 ceph-mon[98643]: Health check failed: Reduced data availability: 2 pgs inactive, 5 pgs peering (PG_AVAILABILITY) 2024-03-24T11:30:26.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:26 smithi062 ceph-mon[98643]: Health check failed: Degraded data redundancy: 14/630 objects degraded (2.222%), 3 pgs degraded (PG_DEGRADED) 2024-03-24T11:30:26.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:26.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:30:26.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:26 smithi062 ceph-mon[90553]: pgmap v260: 161 pgs: 3 active+undersized+degraded, 17 peering, 11 stale+active+clean, 6 active+undersized, 124 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 14/630 objects degraded (2.222%) 2024-03-24T11:30:26.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:26 smithi062 ceph-mon[90553]: Health check failed: Reduced data availability: 2 pgs inactive, 5 pgs peering (PG_AVAILABILITY) 2024-03-24T11:30:26.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:26 smithi062 ceph-mon[90553]: Health check failed: Degraded data redundancy: 14/630 objects degraded (2.222%), 3 pgs degraded (PG_DEGRADED) 2024-03-24T11:30:26.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:26.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:30:26.857 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:30:26 smithi139 podman[131840]: 2024-03-24 11:30:26.72328849 +0000 UTC m=+0.940089191 container remove 5d991a200fbc8948c6ef7f76840d7facff8373204656233cc6e694045be7f3f0 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-24T11:30:26.857 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:30:26 smithi139 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.5.service: Deactivated successfully. 2024-03-24T11:30:26.857 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:30:26 smithi139 systemd[1]: Stopped Ceph osd.5 for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:30:26.858 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:30:26 smithi139 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.5.service: Consumed 11.100s CPU time. 2024-03-24T11:30:26.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:26 smithi139 ceph-mon[91894]: pgmap v260: 161 pgs: 3 active+undersized+degraded, 17 peering, 11 stale+active+clean, 6 active+undersized, 124 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 14/630 objects degraded (2.222%) 2024-03-24T11:30:26.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:26 smithi139 ceph-mon[91894]: Health check failed: Reduced data availability: 2 pgs inactive, 5 pgs peering (PG_AVAILABILITY) 2024-03-24T11:30:26.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:26 smithi139 ceph-mon[91894]: Health check failed: Degraded data redundancy: 14/630 objects degraded (2.222%), 3 pgs degraded (PG_DEGRADED) 2024-03-24T11:30:26.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:26.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:30:27.108 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:30:26 smithi139 systemd[1]: Starting Ceph osd.5 for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:30:28.608 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:30:28 smithi139 podman[132034]: 2024-03-24 11:30:28.449500004 +0000 UTC m=+0.020452334 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:30:28.608 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:30:28 smithi139 podman[132034]: 2024-03-24 11:30:28.568548517 +0000 UTC m=+0.139500878 container create 1b05bd139868b0f487f7a79b1bf9c33cb5ee36fe99fd7645606ffad64477371d (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5-activate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T11:30:28.879 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:28 smithi139 ceph-mon[91894]: pgmap v261: 161 pgs: 11 active+undersized+degraded, 17 peering, 5 stale+active+clean, 19 active+undersized, 109 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 33/630 objects degraded (5.238%) 2024-03-24T11:30:28.880 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:30:28 smithi139 podman[132034]: 2024-03-24 11:30:28.804445115 +0000 UTC m=+0.375397440 container init 1b05bd139868b0f487f7a79b1bf9c33cb5ee36fe99fd7645606ffad64477371d (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=squid-db0330b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2) 2024-03-24T11:30:28.880 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:30:28 smithi139 podman[132034]: 2024-03-24 11:30:28.807745884 +0000 UTC m=+0.378698203 container start 1b05bd139868b0f487f7a79b1bf9c33cb5ee36fe99fd7645606ffad64477371d (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5-activate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-db0330b) 2024-03-24T11:30:28.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:28 smithi062 ceph-mon[90553]: pgmap v261: 161 pgs: 11 active+undersized+degraded, 17 peering, 5 stale+active+clean, 19 active+undersized, 109 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 33/630 objects degraded (5.238%) 2024-03-24T11:30:28.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:28 smithi062 ceph-mon[98643]: pgmap v261: 161 pgs: 11 active+undersized+degraded, 17 peering, 5 stale+active+clean, 19 active+undersized, 109 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 33/630 objects degraded (5.238%) 2024-03-24T11:30:29.283 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:30:28 smithi139 podman[132034]: 2024-03-24 11:30:28.879699731 +0000 UTC m=+0.450652047 container attach 1b05bd139868b0f487f7a79b1bf9c33cb5ee36fe99fd7645606ffad64477371d (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5-activate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-24T11:30:29.599 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:30:29 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5-activate[132076]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-24T11:30:29.599 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:30:29 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5-activate[132076]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-24T11:30:29.599 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:30:29 smithi139 bash[132034]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-24T11:30:29.599 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:30:29 smithi139 bash[132034]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-24T11:30:29.599 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:30:29 smithi139 bash[132034]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-24T11:30:29.599 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:30:29 smithi139 bash[132034]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-24T11:30:29.599 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:30:29 smithi139 bash[132034]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-5/block 2024-03-24T11:30:29.599 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:30:29 smithi139 bash[132034]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-24T11:30:29.599 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:30:29 smithi139 bash[132034]: --> ceph-volume raw activate successful for osd ID: 5 2024-03-24T11:30:29.599 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:30:29 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5-activate[132076]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-24T11:30:29.599 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:30:29 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5-activate[132076]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-24T11:30:29.600 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:30:29 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5-activate[132076]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-5/block 2024-03-24T11:30:29.600 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:30:29 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5-activate[132076]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-24T11:30:29.600 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:30:29 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5-activate[132076]: --> ceph-volume raw activate successful for osd ID: 5 2024-03-24T11:30:29.600 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:30:29 smithi139 podman[132034]: 2024-03-24 11:30:29.390591673 +0000 UTC m=+0.961543998 container died 1b05bd139868b0f487f7a79b1bf9c33cb5ee36fe99fd7645606ffad64477371d (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5-activate, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-db0330b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=) 2024-03-24T11:30:29.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:29.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:29.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:29 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:30.102 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:30.103 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:30.103 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:29 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:30.103 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:30.103 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:30.103 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:29 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:30.671 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:30:30 smithi139 podman[132275]: 2024-03-24 11:30:30.379059758 +0000 UTC m=+1.057998250 container remove 1b05bd139868b0f487f7a79b1bf9c33cb5ee36fe99fd7645606ffad64477371d (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5-activate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-db0330b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-24T11:30:30.930 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:30 smithi139 ceph-mon[91894]: pgmap v262: 161 pgs: 22 active+undersized+degraded, 37 active+undersized, 102 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 74/630 objects degraded (11.746%) 2024-03-24T11:30:30.930 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:30 smithi139 ceph-mon[91894]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 5 pgs peering) 2024-03-24T11:30:30.930 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:30.930 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:30.930 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:31.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:30:30 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:30:30] "GET /metrics HTTP/1.1" 200 38595 "" "Prometheus/2.33.4" 2024-03-24T11:30:31.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:30 smithi062 ceph-mon[98643]: pgmap v262: 161 pgs: 22 active+undersized+degraded, 37 active+undersized, 102 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 74/630 objects degraded (11.746%) 2024-03-24T11:30:31.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:30 smithi062 ceph-mon[98643]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 5 pgs peering) 2024-03-24T11:30:31.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:31.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:31.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:31.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:30 smithi062 ceph-mon[90553]: pgmap v262: 161 pgs: 22 active+undersized+degraded, 37 active+undersized, 102 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 74/630 objects degraded (11.746%) 2024-03-24T11:30:31.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:30 smithi062 ceph-mon[90553]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 5 pgs peering) 2024-03-24T11:30:31.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:31.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:31.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:31.358 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:30:30 smithi139 podman[132414]: 2024-03-24 11:30:30.833074283 +0000 UTC m=+0.347770929 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:30:31.358 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:30:30 smithi139 podman[132414]: 2024-03-24 11:30:30.96264414 +0000 UTC m=+0.477340786 container create 4ea95bfd21272c31051c4598267942aabd9b2ab886bf57b40f01455af009da8c (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-24T11:30:31.858 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:30:31 smithi139 podman[132414]: 2024-03-24 11:30:31.527434624 +0000 UTC m=+1.042131262 container init 4ea95bfd21272c31051c4598267942aabd9b2ab886bf57b40f01455af009da8c (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5, org.label-schema.license=GPLv2, RELEASE=squid-db0330b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-24T11:30:31.858 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:30:31 smithi139 podman[132414]: 2024-03-24 11:30:31.53043218 +0000 UTC m=+1.045128821 container start 4ea95bfd21272c31051c4598267942aabd9b2ab886bf57b40f01455af009da8c (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-24T11:30:31.858 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:30:31 smithi139 ceph-osd[132508]: -- 172.21.15.139:0/826318517 <== mon.2 v2:172.21.15.139:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555ddf6df1e0 con 0x555ddf6b1400 2024-03-24T11:30:31.858 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:30:31 smithi139 bash[132414]: 4ea95bfd21272c31051c4598267942aabd9b2ab886bf57b40f01455af009da8c 2024-03-24T11:30:31.858 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:30:31 smithi139 systemd[1]: Started Ceph osd.5 for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:30:32.317 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:30:32 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5[132460]: 2024-03-24T11:30:32.078+0000 7f4310bac7c0 -1 Falling back to public interface 2024-03-24T11:30:32.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[90553]: pgmap v263: 161 pgs: 22 active+undersized+degraded, 37 active+undersized, 102 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 74/630 objects degraded (11.746%) 2024-03-24T11:30:32.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:32.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:32.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:32.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:32.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:32.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:32.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:32.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:32.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:32.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:32.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:32.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:32.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:32.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:32.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:32.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:32.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:32.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:32.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:32.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:32.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:32.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:32.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:32.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:32.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:32.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:32.980 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:32.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[98643]: pgmap v263: 161 pgs: 22 active+undersized+degraded, 37 active+undersized, 102 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 74/630 objects degraded (11.746%) 2024-03-24T11:30:32.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:32.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:32.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:32.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:32.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:32.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:32.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:32.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:32.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:32.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:32.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:32.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:32.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:32.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:32.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:32.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:32.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:32.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:32.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:32.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:32.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:32.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:32.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:32.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:32.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:32.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:32.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:32 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:33.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:32 smithi139 ceph-mon[91894]: pgmap v263: 161 pgs: 22 active+undersized+degraded, 37 active+undersized, 102 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 74/630 objects degraded (11.746%) 2024-03-24T11:30:33.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:33.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:33.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:33.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:33.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:33.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:33.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:33.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:33.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:33.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:33.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:33.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:33.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:33.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:33.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:33.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:33.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:33.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:33.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:33.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:33.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:33.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:33.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:33.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:33.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:33.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:33.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:32 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:33.689 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:33.689 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:33.689 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:30:33.689 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:33.689 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:33.689 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:33.689 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:33.689 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:33.689 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:33.689 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:33.689 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:33.689 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:33.689 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:33.690 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:33.690 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:33.690 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:33.690 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:33.690 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:33.690 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:33.690 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:33.690 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:33.690 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:33.690 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:33.690 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:33.691 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:33.691 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:33.691 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:33.691 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:33.691 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:33.691 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:33.691 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:33.691 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:33.691 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:33.691 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:33.691 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:33.691 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:33.691 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:33.691 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:33.691 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:33.692 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:33.692 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:33.692 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:33.692 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:33.692 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:33.692 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:33.692 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:33.692 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:33.692 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:33.692 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:30:33.692 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:33.692 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:33.693 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:33.693 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:33.693 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:33.693 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:33.693 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:33.693 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:33.693 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:33.693 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:33.693 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:33.693 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:33.693 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:33.693 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:33.693 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:33.693 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:33.694 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:33.694 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:33.694 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:33.694 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:33.694 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:33.694 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:33.694 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:33.694 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:33.694 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:33.694 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:33.694 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:33.694 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:33.694 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:33.694 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:33.695 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:33.695 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:33.695 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:33.695 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:33.695 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:33.695 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:33.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:33.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:33.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:30:33.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:33.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:33.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:33.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:33.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:33.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:33.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:33.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:33.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:33.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:33.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:33.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:33.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:33.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:33.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:33.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:33.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:33.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:33.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:33.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:33.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:33.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:33.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:33.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:33.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:33.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:33.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:33.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:33.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:33.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:33.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:33.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:33.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:33.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:33.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:33.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:33.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:33.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:33.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:33.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:33.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:33.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:33.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:33.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:33.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:33.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:33.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:33.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:33.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:33.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:33.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:33.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:33.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:33.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:33.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:33.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:33.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:33.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:33.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:33.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:33.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:33.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:33.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:33.864 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:33.864 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:33.864 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:33.864 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:33 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:33.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:33.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:33.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:33.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:33.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:33.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:33.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:33.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:33.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:33.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:33.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:33.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:33.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:33.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:33.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:33.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:33.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:33.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:33.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:33.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:33.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:33.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:33.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:33.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:33.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:33.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:33.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:33.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:33.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:33.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:33.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:33.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:33.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:33.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:33.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:33.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:33.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:33.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:33.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:33.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:33.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:33.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:33.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:33.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:33.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:33.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:33.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:33.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:33.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:33.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:33.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:33.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:33.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:33.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:33.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:33 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:34.809 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:34.809 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:34.809 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:34.809 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:34.809 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:34.809 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: pgmap v264: 161 pgs: 22 active+undersized+degraded, 37 active+undersized, 102 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 74/630 objects degraded (11.746%) 2024-03-24T11:30:34.810 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:34.810 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:34.810 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:34.810 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:34.810 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:34.810 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:34.810 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:34.810 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:34.810 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:34.811 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:34.811 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:34.811 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:34.811 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:34.811 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:34.811 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:34.811 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:34.811 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:34.811 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:34.811 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:34.811 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:34.811 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:34.811 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:34.811 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:34.812 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:34.812 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:34.812 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:34.812 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:34.812 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:34.812 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:34.812 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:34.813 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:34.813 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:34.813 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:34.813 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:34.813 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:34.813 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:34.813 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:34.813 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:34.813 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:34.813 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:34.813 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:34.813 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:34.813 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: Health check update: Degraded data redundancy: 74/630 objects degraded (11.746%), 22 pgs degraded (PG_DEGRADED) 2024-03-24T11:30:34.813 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:34.814 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:34.814 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:34.814 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:34.814 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:34.814 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:34.814 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:34.814 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:34.814 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:34.814 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:34.814 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:34.814 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:34.814 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:34.814 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:34.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:34.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:34.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:34.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:34.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:34.815 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:34.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:34.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:34.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:34.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:34.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:34.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: pgmap v264: 161 pgs: 22 active+undersized+degraded, 37 active+undersized, 102 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 74/630 objects degraded (11.746%) 2024-03-24T11:30:34.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:34.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:34.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:34.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:34.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:34.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:34.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:34.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:34.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:34.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:34.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:34.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:34.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:34.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:34.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:34.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:34.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:34.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:34.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:34.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:34.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:34.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:34.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:34.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:34.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:34.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:34.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:34.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:34.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:34.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: pgmap v264: 161 pgs: 22 active+undersized+degraded, 37 active+undersized, 102 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 74/630 objects degraded (11.746%) 2024-03-24T11:30:34.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:34.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:34.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:34.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:34.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:34.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:34.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:34.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:34.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:34.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:34.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:34.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:34.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:34.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:34.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:34.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:34.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:34.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:34.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:34.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:34.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:34.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:34.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:34.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:34.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:34.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:34.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:34.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:34.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:34.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:34.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:34.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:34.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:34.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:34.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:34.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:34.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:34.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:34.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:34.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:34.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:34.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:34.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: Health check update: Degraded data redundancy: 74/630 objects degraded (11.746%), 22 pgs degraded (PG_DEGRADED) 2024-03-24T11:30:34.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:34.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:34.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:34.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:34.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:34.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:34.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:34.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:34.984 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:34.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:34.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:34.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:34.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:34.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:34.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:34.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:34.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:34.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:34.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:34.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:34.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:34.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:34.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:34.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:34.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:34.985 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:34.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:34.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: Health check update: Degraded data redundancy: 74/630 objects degraded (11.746%), 22 pgs degraded (PG_DEGRADED) 2024-03-24T11:30:34.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:34.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:34.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:34.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:34.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:34.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:34.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:34.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:34.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:34.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:34.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:34.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:34.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:34.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:34.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:34.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:34.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:34.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:34.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:34.987 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:34.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:34.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:34.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:34.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:34.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:34.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:34.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:34.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:34.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:34.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:34.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:34 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:35.108 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5[132460]: 2024-03-24T11:30:34.826+0000 7f4310bac7c0 -1 osd.5 0 read_superblock omap replica is missing. 2024-03-24T11:30:35.108 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:30:34 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5[132460]: 2024-03-24T11:30:34.857+0000 7f4310bac7c0 -1 osd.5 158 log_to_monitors true 2024-03-24T11:30:35.687 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:30:35.309+0000 7f4e4aad7700 -1 mgr.server reply reply (16) Device or resource busy unsafe to stop osd(s) at this time (19 PGs are or would become offline) 2024-03-24T11:30:35.966 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:30:35 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5[132460]: 2024-03-24T11:30:35.821+0000 7f4302004700 -1 osd.5 158 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T11:30:35.967 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:35 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:35.967 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:35 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:35.967 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:35 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:35.967 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:35 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:35.967 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:35 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:35.967 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:35 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:35.967 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:35 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:35.967 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:35.967 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:35.967 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:35.967 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:35.967 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:35.967 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:35 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:35.968 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:35 smithi139 ceph-mon[91894]: from='osd.5 [v2:172.21.15.139:6808/4277451027,v1:172.21.15.139:6809/4277451027]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-24T11:30:35.968 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:35 smithi139 ceph-mon[91894]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-24T11:30:35.968 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:35.968 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:35.968 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:35.968 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:35.968 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:35.968 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:35 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:35.968 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:35.968 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:35.968 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:35.968 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:35.968 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:35.968 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:35 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:35.968 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:35.968 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:35.969 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:35.969 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:35.969 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:35.969 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:35.969 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:35.969 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:35.969 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:35.969 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:35.969 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:35.969 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:35.969 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2024-03-24T11:30:35.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:35.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:35.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:35.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:35.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:35.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:35.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:35.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:35.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:35.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:35.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:35.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:35.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:35.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[90553]: from='osd.5 [v2:172.21.15.139:6808/4277451027,v1:172.21.15.139:6809/4277451027]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-24T11:30:35.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[90553]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-24T11:30:35.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:35.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:35.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:35.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:35.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:35.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:35.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:35.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:35.978 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:35.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:35.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:35.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:35.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:35.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:35.979 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:35.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:35.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:35.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:35.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:35.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:35.979 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:35.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:35.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:35.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:35.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:35.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:35.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:35.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:35.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[98643]: from='osd.5 [v2:172.21.15.139:6808/4277451027,v1:172.21.15.139:6809/4277451027]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-24T11:30:35.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[98643]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-24T11:30:35.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:35.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:35.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:35.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:35.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:35.980 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:35.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:35.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:35.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:35.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:35.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:35.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:35.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:35.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:35.981 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:35.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:35.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:35.981 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:35.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:35.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:35.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:35.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:35.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:35.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:35.982 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2024-03-24T11:30:35.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:30:35.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:35.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:35.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:35.982 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:30:35.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:35.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:35.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:35.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:35.983 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2024-03-24T11:30:37.090 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:36 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:37.090 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:36 smithi139 ceph-mon[91894]: pgmap v265: 161 pgs: 22 active+undersized+degraded, 37 active+undersized, 102 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 74/630 objects degraded (11.746%) 2024-03-24T11:30:37.091 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:36 smithi139 ceph-mon[91894]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2024-03-24T11:30:37.091 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:36 smithi139 ceph-mon[91894]: Upgrade: unsafe to stop osd(s) at this time (19 PGs are or would become offline) 2024-03-24T11:30:37.091 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:36 smithi139 ceph-mon[91894]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-24T11:30:37.091 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:36 smithi139 ceph-mon[91894]: failed to encode map e161 with expected crc 2024-03-24T11:30:37.091 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:36 smithi139 ceph-mon[91894]: osdmap e161: 8 total, 7 up, 8 in 2024-03-24T11:30:37.091 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:36 smithi139 ceph-mon[91894]: from='osd.5 [v2:172.21.15.139:6808/4277451027,v1:172.21.15.139:6809/4277451027]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:30:37.091 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:36 smithi139 ceph-mon[91894]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:30:37.091 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:36 smithi139 ceph-mon[91894]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-03-24T11:30:37.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:37.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:36 smithi062 ceph-mon[90553]: pgmap v265: 161 pgs: 22 active+undersized+degraded, 37 active+undersized, 102 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 74/630 objects degraded (11.746%) 2024-03-24T11:30:37.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:36 smithi062 ceph-mon[90553]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2024-03-24T11:30:37.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:36 smithi062 ceph-mon[90553]: Upgrade: unsafe to stop osd(s) at this time (19 PGs are or would become offline) 2024-03-24T11:30:37.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:36 smithi062 ceph-mon[90553]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-24T11:30:37.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:36 smithi062 ceph-mon[90553]: failed to encode map e161 with expected crc 2024-03-24T11:30:37.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:36 smithi062 ceph-mon[90553]: osdmap e161: 8 total, 7 up, 8 in 2024-03-24T11:30:37.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:36 smithi062 ceph-mon[90553]: from='osd.5 [v2:172.21.15.139:6808/4277451027,v1:172.21.15.139:6809/4277451027]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:30:37.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:36 smithi062 ceph-mon[90553]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:30:37.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:36 smithi062 ceph-mon[90553]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-03-24T11:30:37.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:30:37.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:36 smithi062 ceph-mon[98643]: pgmap v265: 161 pgs: 22 active+undersized+degraded, 37 active+undersized, 102 active+clean; 585 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 74/630 objects degraded (11.746%) 2024-03-24T11:30:37.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:36 smithi062 ceph-mon[98643]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2024-03-24T11:30:37.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:36 smithi062 ceph-mon[98643]: Upgrade: unsafe to stop osd(s) at this time (19 PGs are or would become offline) 2024-03-24T11:30:37.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:36 smithi062 ceph-mon[98643]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-24T11:30:37.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:36 smithi062 ceph-mon[98643]: failed to encode map e161 with expected crc 2024-03-24T11:30:37.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:36 smithi062 ceph-mon[98643]: osdmap e161: 8 total, 7 up, 8 in 2024-03-24T11:30:37.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:36 smithi062 ceph-mon[98643]: from='osd.5 [v2:172.21.15.139:6808/4277451027,v1:172.21.15.139:6809/4277451027]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:30:37.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:36 smithi062 ceph-mon[98643]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:30:37.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:36 smithi062 ceph-mon[98643]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-03-24T11:30:38.055 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:37 smithi139 ceph-mon[91894]: failed to encode map e161 with expected crc 2024-03-24T11:30:38.056 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:37 smithi139 ceph-mon[91894]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T11:30:38.056 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:37 smithi139 ceph-mon[91894]: failed to encode map e162 with expected crc 2024-03-24T11:30:38.056 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:37 smithi139 ceph-mon[91894]: osd.5 [v2:172.21.15.139:6808/4277451027,v1:172.21.15.139:6809/4277451027] boot 2024-03-24T11:30:38.056 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:37 smithi139 ceph-mon[91894]: osdmap e162: 8 total, 8 up, 8 in 2024-03-24T11:30:38.056 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:37 smithi139 ceph-mon[91894]: failed to encode map e162 with expected crc 2024-03-24T11:30:38.056 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T11:30:38.056 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:37 smithi139 ceph-mon[91894]: failed to encode map e162 with expected crc 2024-03-24T11:30:38.056 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:37 smithi139 ceph-mon[91894]: failed to encode map e162 with expected crc 2024-03-24T11:30:38.056 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:37 smithi139 ceph-mon[91894]: osdmap e163: 8 total, 8 up, 8 in 2024-03-24T11:30:38.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:37 smithi062 ceph-mon[90553]: failed to encode map e161 with expected crc 2024-03-24T11:30:38.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:37 smithi062 ceph-mon[90553]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T11:30:38.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:37 smithi062 ceph-mon[90553]: failed to encode map e162 with expected crc 2024-03-24T11:30:38.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:37 smithi062 ceph-mon[90553]: osd.5 [v2:172.21.15.139:6808/4277451027,v1:172.21.15.139:6809/4277451027] boot 2024-03-24T11:30:38.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:37 smithi062 ceph-mon[90553]: osdmap e162: 8 total, 8 up, 8 in 2024-03-24T11:30:38.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:37 smithi062 ceph-mon[90553]: failed to encode map e162 with expected crc 2024-03-24T11:30:38.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T11:30:38.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:37 smithi062 ceph-mon[90553]: failed to encode map e162 with expected crc 2024-03-24T11:30:38.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:37 smithi062 ceph-mon[90553]: failed to encode map e162 with expected crc 2024-03-24T11:30:38.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:37 smithi062 ceph-mon[90553]: osdmap e163: 8 total, 8 up, 8 in 2024-03-24T11:30:38.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:37 smithi062 ceph-mon[98643]: failed to encode map e161 with expected crc 2024-03-24T11:30:38.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:37 smithi062 ceph-mon[98643]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T11:30:38.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:37 smithi062 ceph-mon[98643]: failed to encode map e162 with expected crc 2024-03-24T11:30:38.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:37 smithi062 ceph-mon[98643]: osd.5 [v2:172.21.15.139:6808/4277451027,v1:172.21.15.139:6809/4277451027] boot 2024-03-24T11:30:38.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:37 smithi062 ceph-mon[98643]: osdmap e162: 8 total, 8 up, 8 in 2024-03-24T11:30:38.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:37 smithi062 ceph-mon[98643]: failed to encode map e162 with expected crc 2024-03-24T11:30:38.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T11:30:38.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:37 smithi062 ceph-mon[98643]: failed to encode map e162 with expected crc 2024-03-24T11:30:38.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:37 smithi062 ceph-mon[98643]: failed to encode map e162 with expected crc 2024-03-24T11:30:38.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:37 smithi062 ceph-mon[98643]: osdmap e163: 8 total, 8 up, 8 in 2024-03-24T11:30:39.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:38 smithi139 ceph-mon[91894]: pgmap v268: 161 pgs: 10 peering, 19 active+undersized+degraded, 30 active+undersized, 102 active+clean; 585 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 652 B/s rd, 0 op/s; 63/630 objects degraded (10.000%) 2024-03-24T11:30:39.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:38 smithi139 ceph-mon[91894]: failed to encode map e163 with expected crc 2024-03-24T11:30:39.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:38 smithi139 ceph-mon[91894]: failed to encode map e163 with expected crc 2024-03-24T11:30:39.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:38 smithi139 ceph-mon[91894]: failed to encode map e163 with expected crc 2024-03-24T11:30:39.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:38 smithi139 ceph-mon[91894]: failed to encode map e163 with expected crc 2024-03-24T11:30:39.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:38 smithi062 ceph-mon[90553]: pgmap v268: 161 pgs: 10 peering, 19 active+undersized+degraded, 30 active+undersized, 102 active+clean; 585 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 652 B/s rd, 0 op/s; 63/630 objects degraded (10.000%) 2024-03-24T11:30:39.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:38 smithi062 ceph-mon[90553]: failed to encode map e163 with expected crc 2024-03-24T11:30:39.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:38 smithi062 ceph-mon[90553]: failed to encode map e163 with expected crc 2024-03-24T11:30:39.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:38 smithi062 ceph-mon[90553]: failed to encode map e163 with expected crc 2024-03-24T11:30:39.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:38 smithi062 ceph-mon[90553]: failed to encode map e163 with expected crc 2024-03-24T11:30:39.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:38 smithi062 ceph-mon[98643]: pgmap v268: 161 pgs: 10 peering, 19 active+undersized+degraded, 30 active+undersized, 102 active+clean; 585 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 652 B/s rd, 0 op/s; 63/630 objects degraded (10.000%) 2024-03-24T11:30:39.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:38 smithi062 ceph-mon[98643]: failed to encode map e163 with expected crc 2024-03-24T11:30:39.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:38 smithi062 ceph-mon[98643]: failed to encode map e163 with expected crc 2024-03-24T11:30:39.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:38 smithi062 ceph-mon[98643]: failed to encode map e163 with expected crc 2024-03-24T11:30:39.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:38 smithi062 ceph-mon[98643]: failed to encode map e163 with expected crc 2024-03-24T11:30:40.090 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:39 smithi139 ceph-mon[91894]: Health check update: Degraded data redundancy: 63/630 objects degraded (10.000%), 19 pgs degraded (PG_DEGRADED) 2024-03-24T11:30:40.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:39 smithi062 ceph-mon[90553]: Health check update: Degraded data redundancy: 63/630 objects degraded (10.000%), 19 pgs degraded (PG_DEGRADED) 2024-03-24T11:30:40.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:39 smithi062 ceph-mon[98643]: Health check update: Degraded data redundancy: 63/630 objects degraded (10.000%), 19 pgs degraded (PG_DEGRADED) 2024-03-24T11:30:41.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:40 smithi139 ceph-mon[91894]: pgmap v270: 161 pgs: 10 peering, 12 active+undersized+degraded, 20 active+undersized, 119 active+clean; 585 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s; 36/630 objects degraded (5.714%) 2024-03-24T11:30:41.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:40 smithi062 ceph-mon[90553]: pgmap v270: 161 pgs: 10 peering, 12 active+undersized+degraded, 20 active+undersized, 119 active+clean; 585 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s; 36/630 objects degraded (5.714%) 2024-03-24T11:30:41.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:40 smithi062 ceph-mon[98643]: pgmap v270: 161 pgs: 10 peering, 12 active+undersized+degraded, 20 active+undersized, 119 active+clean; 585 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s; 36/630 objects degraded (5.714%) 2024-03-24T11:30:41.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:30:40 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:30:40] "GET /metrics HTTP/1.1" 200 38595 "" "Prometheus/2.33.4" 2024-03-24T11:30:42.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:42.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:30:42.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:42 smithi139 ceph-mon[91894]: pgmap v271: 161 pgs: 10 peering, 1 active+undersized+degraded, 4 active+undersized, 146 active+clean; 585 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s; 1/630 objects degraded (0.159%) 2024-03-24T11:30:42.427 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:42.427 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:30:42.427 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:42 smithi062 ceph-mon[90553]: pgmap v271: 161 pgs: 10 peering, 1 active+undersized+degraded, 4 active+undersized, 146 active+clean; 585 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s; 1/630 objects degraded (0.159%) 2024-03-24T11:30:42.428 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:42.428 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:30:42.428 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:42 smithi062 ceph-mon[98643]: pgmap v271: 161 pgs: 10 peering, 1 active+undersized+degraded, 4 active+undersized, 146 active+clean; 585 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s; 1/630 objects degraded (0.159%) 2024-03-24T11:30:42.554 INFO:teuthology.orchestra.run.smithi062.stdout:true 2024-03-24T11:30:43.222 INFO:teuthology.orchestra.run.smithi062.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T11:30:43.222 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi062 smithi062 running 13s ago 20m - - 2024-03-24T11:30:43.222 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi139 smithi139 running 7s ago 19m - - 2024-03-24T11:30:43.223 INFO:teuthology.orchestra.run.smithi062.stdout:alertmanager.a smithi062 *:9093,9094 running (9m) 13s ago 14m 25.5M - 0.23.0 ba2b418f427c 818298d58140 2024-03-24T11:30:43.223 INFO:teuthology.orchestra.run.smithi062.stdout:grafana.a smithi139 *:3000 running (9m) 7s ago 14m 54.5M - 8.3.5 dad864ee21e9 b48167e7171e 2024-03-24T11:30:43.223 INFO:teuthology.orchestra.run.smithi062.stdout:iscsi.foo.smithi062.dggsvk smithi062 running (8m) 13s ago 13m 66.2M - 3.5 e1d6a67b021e 0a98806ec839 2024-03-24T11:30:43.223 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.x smithi139 *:8443,9283,8765 running (7m) 7s ago 19m 452M - 19.0.0-1667-gdb0330b1 4f325f8440e9 d94987b11387 2024-03-24T11:30:43.223 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.y smithi062 *:8443,9283,8765 running (8m) 13s ago 21m 532M - 19.0.0-1667-gdb0330b1 4f325f8440e9 ca1a49028d65 2024-03-24T11:30:43.223 INFO:teuthology.orchestra.run.smithi062.stdout:mon.a smithi062 running (6m) 13s ago 21m 70.6M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 f7afe6e28c88 2024-03-24T11:30:43.223 INFO:teuthology.orchestra.run.smithi062.stdout:mon.b smithi139 running (5m) 7s ago 19m 50.6M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 ca357beea4dd 2024-03-24T11:30:43.223 INFO:teuthology.orchestra.run.smithi062.stdout:mon.c smithi062 running (6m) 13s ago 19m 60.0M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 459e7449ec39 2024-03-24T11:30:43.223 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.a smithi062 *:9100 running (10m) 13s ago 15m 15.3M - 1.3.1 1dbe0e931976 79d5a02a999f 2024-03-24T11:30:43.223 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.b smithi139 *:9100 running (9m) 7s ago 15m 18.9M - 1.3.1 1dbe0e931976 8f9f4608e465 2024-03-24T11:30:43.223 INFO:teuthology.orchestra.run.smithi062.stdout:osd.0 smithi062 running (4m) 13s ago 19m 60.0M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 dfe4920b49c4 2024-03-24T11:30:43.223 INFO:teuthology.orchestra.run.smithi062.stdout:osd.1 smithi062 running (3m) 13s ago 18m 52.1M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 c443b3fe97c1 2024-03-24T11:30:43.223 INFO:teuthology.orchestra.run.smithi062.stdout:osd.2 smithi062 running (2m) 13s ago 18m 46.8M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 0eba40dc96b3 2024-03-24T11:30:43.223 INFO:teuthology.orchestra.run.smithi062.stdout:osd.3 smithi062 running (104s) 13s ago 17m 55.1M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 7501d2e4c718 2024-03-24T11:30:43.223 INFO:teuthology.orchestra.run.smithi062.stdout:osd.4 smithi139 running (48s) 7s ago 17m 50.2M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 5d0f7f05ec6d 2024-03-24T11:30:43.223 INFO:teuthology.orchestra.run.smithi062.stdout:osd.5 smithi139 unknown 7s ago 17m - 4096M 2024-03-24T11:30:43.224 INFO:teuthology.orchestra.run.smithi062.stdout:osd.6 smithi139 running (16m) 7s ago 16m 65.2M 4096M 17.2.0 e1d6a67b021e a4b1289aca69 2024-03-24T11:30:43.224 INFO:teuthology.orchestra.run.smithi062.stdout:osd.7 smithi139 running (16m) 7s ago 16m 67.5M 4096M 17.2.0 e1d6a67b021e bc7fadbc1384 2024-03-24T11:30:43.224 INFO:teuthology.orchestra.run.smithi062.stdout:prometheus.a smithi139 *:9095 running (8m) 7s ago 15m 71.3M - 2.33.4 514e6a882f6e 2ed47f810d33 2024-03-24T11:30:43.224 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi062.sacpds smithi062 *:8000 running (13m) 13s ago 13m 75.1M - 17.2.0 e1d6a67b021e 89f4ac308e98 2024-03-24T11:30:43.224 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi139.yljesh smithi139 *:8000 running (13m) 7s ago 13m 74.1M - 17.2.0 e1d6a67b021e e189bdc12cae 2024-03-24T11:30:43.224 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi062.xvpxnk smithi062 *:80 running (13m) 13s ago 13m 74.9M - 17.2.0 e1d6a67b021e d1aa74adab44 2024-03-24T11:30:43.224 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi139.pbtrmp smithi139 *:80 running (13m) 7s ago 13m 78.0M - 17.2.0 e1d6a67b021e 7f8e336755d7 2024-03-24T11:30:43.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:43 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:30:43.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:43 smithi062 ceph-mon[90553]: from='client.44589 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:30:43.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:43 smithi062 ceph-mon[90553]: from='client.64564 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:30:43.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:43 smithi062 ceph-mon[98643]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:30:43.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:43 smithi062 ceph-mon[98643]: from='client.44589 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:30:43.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:43 smithi062 ceph-mon[98643]: from='client.64564 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:30:43.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:43 smithi139 ceph-mon[91894]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:30:43.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:43 smithi139 ceph-mon[91894]: from='client.44589 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:30:43.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:43 smithi139 ceph-mon[91894]: from='client.64564 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:30:43.636 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:30:43.636 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": { 2024-03-24T11:30:43.636 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 3 2024-03-24T11:30:43.636 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:30:43.636 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": { 2024-03-24T11:30:43.636 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-24T11:30:43.636 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:30:43.636 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": { 2024-03-24T11:30:43.636 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 2, 2024-03-24T11:30:43.636 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 6 2024-03-24T11:30:43.637 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:30:43.637 INFO:teuthology.orchestra.run.smithi062.stdout: "rgw": { 2024-03-24T11:30:43.637 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4 2024-03-24T11:30:43.637 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:30:43.637 INFO:teuthology.orchestra.run.smithi062.stdout: "overall": { 2024-03-24T11:30:43.637 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 6, 2024-03-24T11:30:43.637 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 11 2024-03-24T11:30:43.637 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-24T11:30:43.637 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:30:43.988 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:30:43.989 INFO:teuthology.orchestra.run.smithi062.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69", 2024-03-24T11:30:43.989 INFO:teuthology.orchestra.run.smithi062.stdout: "in_progress": true, 2024-03-24T11:30:43.989 INFO:teuthology.orchestra.run.smithi062.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T11:30:43.989 INFO:teuthology.orchestra.run.smithi062.stdout: "services_complete": [ 2024-03-24T11:30:43.989 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr", 2024-03-24T11:30:43.989 INFO:teuthology.orchestra.run.smithi062.stdout: "mon" 2024-03-24T11:30:43.989 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-03-24T11:30:43.989 INFO:teuthology.orchestra.run.smithi062.stdout: "progress": "11/23 daemons upgraded", 2024-03-24T11:30:43.989 INFO:teuthology.orchestra.run.smithi062.stdout: "message": "Currently upgrading osd daemons", 2024-03-24T11:30:43.989 INFO:teuthology.orchestra.run.smithi062.stdout: "is_paused": false 2024-03-24T11:30:43.989 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:30:44.403 INFO:teuthology.orchestra.run.smithi062.stdout:HEALTH_OK 2024-03-24T11:30:44.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:44 smithi062 ceph-mon[90553]: from='client.44601 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:30:44.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:44 smithi062 ceph-mon[90553]: pgmap v272: 161 pgs: 161 active+clean; 585 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 409 B/s rd, 0 op/s 2024-03-24T11:30:44.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:44.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:44.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:44 smithi062 ceph-mon[90553]: pgmap v273: 161 pgs: 161 active+clean; 585 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 465 B/s rd, 0 op/s 2024-03-24T11:30:44.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:44.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:44.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:44.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:44.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:44 smithi062 ceph-mon[90553]: from='client.? 172.21.15.62:0/3831611850' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:44.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:44 smithi062 ceph-mon[90553]: from='client.64582 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:30:44.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:44 smithi062 ceph-mon[90553]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-03-24T11:30:44.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:44 smithi062 ceph-mon[90553]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 1/630 objects degraded (0.159%), 1 pg degraded) 2024-03-24T11:30:44.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:44 smithi062 ceph-mon[90553]: Cluster is now healthy 2024-03-24T11:30:44.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:44 smithi062 ceph-mon[98643]: from='client.44601 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:30:44.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:44 smithi062 ceph-mon[98643]: pgmap v272: 161 pgs: 161 active+clean; 585 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 409 B/s rd, 0 op/s 2024-03-24T11:30:44.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:44.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:44.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:44 smithi062 ceph-mon[98643]: pgmap v273: 161 pgs: 161 active+clean; 585 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 465 B/s rd, 0 op/s 2024-03-24T11:30:44.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:44.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:44.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:44.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:44.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:44 smithi062 ceph-mon[98643]: from='client.? 172.21.15.62:0/3831611850' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:44.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:44 smithi062 ceph-mon[98643]: from='client.64582 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:30:44.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:44 smithi062 ceph-mon[98643]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-03-24T11:30:44.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:44 smithi062 ceph-mon[98643]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 1/630 objects degraded (0.159%), 1 pg degraded) 2024-03-24T11:30:44.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:44 smithi062 ceph-mon[98643]: Cluster is now healthy 2024-03-24T11:30:44.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:44 smithi139 ceph-mon[91894]: from='client.44601 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:30:44.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:44 smithi139 ceph-mon[91894]: pgmap v272: 161 pgs: 161 active+clean; 585 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 409 B/s rd, 0 op/s 2024-03-24T11:30:44.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:44.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:44.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:44 smithi139 ceph-mon[91894]: pgmap v273: 161 pgs: 161 active+clean; 585 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 465 B/s rd, 0 op/s 2024-03-24T11:30:44.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:44.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:44.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:44.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:44.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:44 smithi139 ceph-mon[91894]: from='client.? 172.21.15.62:0/3831611850' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:30:44.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:44 smithi139 ceph-mon[91894]: from='client.64582 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:30:44.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:44 smithi139 ceph-mon[91894]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-03-24T11:30:44.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:44 smithi139 ceph-mon[91894]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 1/630 objects degraded (0.159%), 1 pg degraded) 2024-03-24T11:30:44.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:44 smithi139 ceph-mon[91894]: Cluster is now healthy 2024-03-24T11:30:45.408 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:45 smithi062 ceph-mon[90553]: from='client.? 172.21.15.62:0/399877380' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:30:45.409 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:45 smithi062 ceph-mon[98643]: from='client.? 172.21.15.62:0/399877380' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:30:45.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:45 smithi139 ceph-mon[91894]: from='client.? 172.21.15.62:0/399877380' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:30:46.673 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:46 smithi062 ceph-mon[90553]: pgmap v274: 161 pgs: 161 active+clean; 585 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 1011 B/s rd, 0 op/s 2024-03-24T11:30:46.673 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:46 smithi062 ceph-mon[98643]: pgmap v274: 161 pgs: 161 active+clean; 585 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 1011 B/s rd, 0 op/s 2024-03-24T11:30:46.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:46 smithi139 ceph-mon[91894]: pgmap v274: 161 pgs: 161 active+clean; 585 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 1011 B/s rd, 0 op/s 2024-03-24T11:30:48.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:48 smithi062 ceph-mon[90553]: pgmap v275: 161 pgs: 161 active+clean; 585 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:30:48.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:48 smithi062 ceph-mon[98643]: pgmap v275: 161 pgs: 161 active+clean; 585 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:30:48.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:48 smithi139 ceph-mon[91894]: pgmap v275: 161 pgs: 161 active+clean; 585 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:30:50.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:50 smithi062 ceph-mon[90553]: pgmap v276: 161 pgs: 161 active+clean; 585 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 912 B/s rd, 0 op/s 2024-03-24T11:30:50.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:50 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:50.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:50 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:50.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:50 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:50.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:50 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2024-03-24T11:30:50.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:50 smithi062 ceph-mon[98643]: pgmap v276: 161 pgs: 161 active+clean; 585 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 912 B/s rd, 0 op/s 2024-03-24T11:30:50.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:50 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:50.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:50 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:50.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:50 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:50.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:50 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2024-03-24T11:30:50.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:50 smithi139 ceph-mon[91894]: pgmap v276: 161 pgs: 161 active+clean; 585 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 912 B/s rd, 0 op/s 2024-03-24T11:30:50.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:50 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:50.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:50 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:50.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:50 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:50.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:50 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2024-03-24T11:30:51.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:30:50 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:30:50] "GET /metrics HTTP/1.1" 200 38603 "" "Prometheus/2.33.4" 2024-03-24T11:30:51.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:51 smithi139 ceph-mon[91894]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2024-03-24T11:30:51.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:51 smithi139 ceph-mon[91894]: Upgrade: osd.6 is safe to restart 2024-03-24T11:30:51.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:51 smithi062 ceph-mon[90553]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2024-03-24T11:30:51.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:51 smithi062 ceph-mon[90553]: Upgrade: osd.6 is safe to restart 2024-03-24T11:30:51.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:51 smithi062 ceph-mon[98643]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2024-03-24T11:30:51.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:51 smithi062 ceph-mon[98643]: Upgrade: osd.6 is safe to restart 2024-03-24T11:30:52.718 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:52 smithi139 ceph-mon[91894]: pgmap v277: 161 pgs: 161 active+clean; 585 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 1014 B/s rd, 0 op/s 2024-03-24T11:30:52.718 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:52 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:52.718 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:52 smithi139 ceph-mon[91894]: Upgrade: Updating osd.6 2024-03-24T11:30:52.719 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:52 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:52.719 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:52 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-24T11:30:52.719 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:52 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:52.719 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:52 smithi139 ceph-mon[91894]: Deploying daemon osd.6 on smithi139 2024-03-24T11:30:52.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:52 smithi062 ceph-mon[90553]: pgmap v277: 161 pgs: 161 active+clean; 585 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 1014 B/s rd, 0 op/s 2024-03-24T11:30:52.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:52 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:52.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:52 smithi062 ceph-mon[90553]: Upgrade: Updating osd.6 2024-03-24T11:30:52.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:52 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:52.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:52 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-24T11:30:52.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:52 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:52.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:52 smithi062 ceph-mon[90553]: Deploying daemon osd.6 on smithi139 2024-03-24T11:30:52.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:52 smithi062 ceph-mon[98643]: pgmap v277: 161 pgs: 161 active+clean; 585 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 1014 B/s rd, 0 op/s 2024-03-24T11:30:52.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:52 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:52.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:52 smithi062 ceph-mon[98643]: Upgrade: Updating osd.6 2024-03-24T11:30:52.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:52 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:52.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:52 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-24T11:30:52.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:52 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:30:52.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:52 smithi062 ceph-mon[98643]: Deploying daemon osd.6 on smithi139 2024-03-24T11:30:53.784 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:53 smithi139 ceph-mon[91894]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:30:53.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:53 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:30:53.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:53 smithi062 ceph-mon[98643]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:30:54.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:54 smithi139 ceph-mon[91894]: pgmap v278: 161 pgs: 161 active+clean; 585 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 1014 B/s rd, 0 op/s 2024-03-24T11:30:54.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:54 smithi062 ceph-mon[90553]: pgmap v278: 161 pgs: 161 active+clean; 585 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 1014 B/s rd, 0 op/s 2024-03-24T11:30:54.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:54 smithi062 ceph-mon[98643]: pgmap v278: 161 pgs: 161 active+clean; 585 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 1014 B/s rd, 0 op/s 2024-03-24T11:30:55.513 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:30:55 smithi139 systemd[1]: Stopping Ceph osd.6 for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:30:55.858 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:30:55 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[36139]: 2024-03-24T11:30:55.512+0000 7f4ab63e1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T11:30:55.858 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:30:55 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[36139]: 2024-03-24T11:30:55.512+0000 7f4ab63e1700 -1 osd.6 163 *** Got signal Terminated *** 2024-03-24T11:30:55.858 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:30:55 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[36139]: 2024-03-24T11:30:55.512+0000 7f4ab63e1700 -1 osd.6 163 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T11:30:56.779 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:56 smithi062 ceph-mon[98643]: pgmap v279: 161 pgs: 161 active+clean; 585 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:30:56.779 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:56 smithi062 ceph-mon[98643]: osd.6 marked itself down and dead 2024-03-24T11:30:56.779 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:56.779 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:30:56.780 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:56 smithi062 ceph-mon[90553]: pgmap v279: 161 pgs: 161 active+clean; 585 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:30:56.780 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:56 smithi062 ceph-mon[90553]: osd.6 marked itself down and dead 2024-03-24T11:30:56.780 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:56.780 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:30:56.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:56 smithi139 ceph-mon[91894]: pgmap v279: 161 pgs: 161 active+clean; 585 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:30:56.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:56 smithi139 ceph-mon[91894]: osd.6 marked itself down and dead 2024-03-24T11:30:56.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:30:56.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:30:57.263 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:30:56 smithi139 podman[135581]: 2024-03-24 11:30:56.963619754 +0000 UTC m=+1.566941418 container died a4b1289aca69559effa15cfc876ffbf66d018e3d6d0709a49e0854f4ec23ae4f (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., ceph=True, io.buildah.version=1.19.8, distribution-scope=public, maintainer=Guillaume Abrioux , architecture=x86_64, version=8, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, GIT_CLEAN=True, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, name=centos-stream, io.openshift.expose-services=, vcs-type=git, GIT_BRANCH=HEAD, com.redhat.component=centos-stream-container, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, io.k8s.display-name=CentOS Stream 8, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, GIT_REPO=https://github.com/ceph/ceph-container.git, vendor=Red Hat, Inc., com.redhat.license_terms=https://centos.org/legal/licensing-policy/, RELEASE=HEAD, release=754, CEPH_POINT_RELEASE=-17.2.0, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, io.openshift.tags=base centos centos-stream, build-date=2022-05-03T08:36:31.336870) 2024-03-24T11:30:57.605 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:30:57 smithi139 podman[135581]: 2024-03-24 11:30:57.362631777 +0000 UTC m=+1.965953447 container cleanup a4b1289aca69559effa15cfc876ffbf66d018e3d6d0709a49e0854f4ec23ae4f (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6, GIT_REPO=https://github.com/ceph/ceph-container.git, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, CEPH_POINT_RELEASE=-17.2.0, name=centos-stream, release=754, GIT_BRANCH=HEAD, io.buildah.version=1.19.8, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, io.openshift.tags=base centos centos-stream, RELEASE=HEAD, maintainer=Guillaume Abrioux , io.k8s.display-name=CentOS Stream 8, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.openshift.expose-services=, vcs-type=git, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, architecture=x86_64, build-date=2022-05-03T08:36:31.336870, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, version=8, vendor=Red Hat, Inc., GIT_CLEAN=True, com.redhat.component=centos-stream-container, distribution-scope=public, ceph=True) 2024-03-24T11:30:57.606 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:30:57 smithi139 bash[135581]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6 2024-03-24T11:30:57.606 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:57 smithi139 ceph-mon[91894]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T11:30:57.606 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:57 smithi139 ceph-mon[91894]: failed to encode map e164 with expected crc 2024-03-24T11:30:57.606 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:57 smithi139 ceph-mon[91894]: osdmap e164: 8 total, 7 up, 8 in 2024-03-24T11:30:57.606 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:57 smithi139 ceph-mon[91894]: failed to encode map e164 with expected crc 2024-03-24T11:30:57.606 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:57 smithi139 ceph-mon[91894]: failed to encode map e164 with expected crc 2024-03-24T11:30:57.795 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:57 smithi062 ceph-mon[90553]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T11:30:57.795 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:57 smithi062 ceph-mon[90553]: failed to encode map e164 with expected crc 2024-03-24T11:30:57.795 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:57 smithi062 ceph-mon[90553]: osdmap e164: 8 total, 7 up, 8 in 2024-03-24T11:30:57.795 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:57 smithi062 ceph-mon[90553]: failed to encode map e164 with expected crc 2024-03-24T11:30:57.795 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:57 smithi062 ceph-mon[90553]: failed to encode map e164 with expected crc 2024-03-24T11:30:57.795 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:57 smithi062 ceph-mon[98643]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T11:30:57.795 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:57 smithi062 ceph-mon[98643]: failed to encode map e164 with expected crc 2024-03-24T11:30:57.796 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:57 smithi062 ceph-mon[98643]: osdmap e164: 8 total, 7 up, 8 in 2024-03-24T11:30:57.796 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:57 smithi062 ceph-mon[98643]: failed to encode map e164 with expected crc 2024-03-24T11:30:57.796 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:57 smithi062 ceph-mon[98643]: failed to encode map e164 with expected crc 2024-03-24T11:30:57.858 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:30:57 smithi139 podman[135598]: 2024-03-24 11:30:57.605673773 +0000 UTC m=+0.641439898 container remove a4b1289aca69559effa15cfc876ffbf66d018e3d6d0709a49e0854f4ec23ae4f (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6, vendor=Red Hat, Inc., GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, maintainer=Guillaume Abrioux , url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, RELEASE=HEAD, io.openshift.tags=base centos centos-stream, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, version=8, ceph=True, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, vcs-type=git, io.openshift.expose-services=, release=754, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., build-date=2022-05-03T08:36:31.336870, io.buildah.version=1.19.8, CEPH_POINT_RELEASE=-17.2.0, com.redhat.component=centos-stream-container, name=centos-stream, distribution-scope=public, architecture=x86_64, GIT_CLEAN=True, io.k8s.display-name=CentOS Stream 8, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb) 2024-03-24T11:30:58.788 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:58 smithi139 ceph-mon[91894]: pgmap v281: 161 pgs: 9 peering, 13 stale+active+clean, 139 active+clean; 585 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T11:30:58.788 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:58 smithi139 ceph-mon[91894]: failed to encode map e165 with expected crc 2024-03-24T11:30:58.788 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:58 smithi139 ceph-mon[91894]: failed to encode map e165 with expected crc 2024-03-24T11:30:58.788 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:58 smithi139 ceph-mon[91894]: osdmap e165: 8 total, 7 up, 8 in 2024-03-24T11:30:58.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:58 smithi062 ceph-mon[90553]: pgmap v281: 161 pgs: 9 peering, 13 stale+active+clean, 139 active+clean; 585 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T11:30:58.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:58 smithi062 ceph-mon[90553]: failed to encode map e165 with expected crc 2024-03-24T11:30:58.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:58 smithi062 ceph-mon[90553]: failed to encode map e165 with expected crc 2024-03-24T11:30:58.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:58 smithi062 ceph-mon[90553]: osdmap e165: 8 total, 7 up, 8 in 2024-03-24T11:30:58.966 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:58 smithi062 ceph-mon[98643]: pgmap v281: 161 pgs: 9 peering, 13 stale+active+clean, 139 active+clean; 585 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T11:30:58.966 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:58 smithi062 ceph-mon[98643]: failed to encode map e165 with expected crc 2024-03-24T11:30:58.966 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:58 smithi062 ceph-mon[98643]: failed to encode map e165 with expected crc 2024-03-24T11:30:58.966 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:58 smithi062 ceph-mon[98643]: osdmap e165: 8 total, 7 up, 8 in 2024-03-24T11:30:59.047 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:30:59 smithi139 podman[135845]: 2024-03-24 11:30:58.930286658 +0000 UTC m=+0.018648484 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:30:59.358 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:30:59 smithi139 podman[135845]: 2024-03-24 11:30:59.047090692 +0000 UTC m=+0.135452561 container create b64eb67bd71c4c7c07eba71e60d7df65a12ac94f15b05d354864f72b1aa0ef65 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-db0330b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-24T11:30:59.358 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:30:59 smithi139 podman[135845]: 2024-03-24 11:30:59.274330774 +0000 UTC m=+0.362692601 container init b64eb67bd71c4c7c07eba71e60d7df65a12ac94f15b05d354864f72b1aa0ef65 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-db0330b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-24T11:30:59.358 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:30:59 smithi139 podman[135845]: 2024-03-24 11:30:59.278311087 +0000 UTC m=+0.366672922 container start b64eb67bd71c4c7c07eba71e60d7df65a12ac94f15b05d354864f72b1aa0ef65 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-db0330b, org.label-schema.license=GPLv2) 2024-03-24T11:30:59.678 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:30:59 smithi139 ceph-mon[91894]: Health check failed: Degraded data redundancy: 22/630 objects degraded (3.492%), 5 pgs degraded (PG_DEGRADED) 2024-03-24T11:30:59.678 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:30:59 smithi139 podman[135845]: 2024-03-24 11:30:59.384423136 +0000 UTC m=+0.472784988 container attach b64eb67bd71c4c7c07eba71e60d7df65a12ac94f15b05d354864f72b1aa0ef65 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, RELEASE=squid-db0330b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T11:30:59.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:30:59 smithi062 ceph-mon[90553]: Health check failed: Degraded data redundancy: 22/630 objects degraded (3.492%), 5 pgs degraded (PG_DEGRADED) 2024-03-24T11:30:59.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:30:59 smithi062 ceph-mon[98643]: Health check failed: Degraded data redundancy: 22/630 objects degraded (3.492%), 5 pgs degraded (PG_DEGRADED) 2024-03-24T11:31:00.003 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:30:59 smithi139 podman[135845]: 2024-03-24 11:30:59.678309403 +0000 UTC m=+0.766671221 container died b64eb67bd71c4c7c07eba71e60d7df65a12ac94f15b05d354864f72b1aa0ef65 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-db0330b) 2024-03-24T11:31:00.740 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:00 smithi139 ceph-mon[91894]: pgmap v283: 161 pgs: 5 active+undersized+degraded, 9 peering, 10 stale+active+clean, 13 active+undersized, 124 active+clean; 585 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 22/630 objects degraded (3.492%) 2024-03-24T11:31:00.805 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:00 smithi062 ceph-mon[90553]: pgmap v283: 161 pgs: 5 active+undersized+degraded, 9 peering, 10 stale+active+clean, 13 active+undersized, 124 active+clean; 585 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 22/630 objects degraded (3.492%) 2024-03-24T11:31:00.805 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:00 smithi062 ceph-mon[98643]: pgmap v283: 161 pgs: 5 active+undersized+degraded, 9 peering, 10 stale+active+clean, 13 active+undersized, 124 active+clean; 585 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 22/630 objects degraded (3.492%) 2024-03-24T11:31:01.108 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:31:00 smithi139 podman[135932]: 2024-03-24 11:31:00.740435297 +0000 UTC m=+1.135930448 container remove b64eb67bd71c4c7c07eba71e60d7df65a12ac94f15b05d354864f72b1aa0ef65 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6-deactivate, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T11:31:01.108 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:31:00 smithi139 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.6.service: Deactivated successfully. 2024-03-24T11:31:01.108 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:31:00 smithi139 systemd[1]: Stopped Ceph osd.6 for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:31:01.108 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:31:00 smithi139 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.6.service: Consumed 10.023s CPU time. 2024-03-24T11:31:01.108 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:31:00 smithi139 systemd[1]: Starting Ceph osd.6 for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:31:01.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:31:00 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:31:00] "GET /metrics HTTP/1.1" 200 38608 "" "Prometheus/2.33.4" 2024-03-24T11:31:01.580 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:31:01 smithi139 podman[136126]: 2024-03-24 11:31:01.095033695 +0000 UTC m=+0.018477788 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:31:01.580 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:31:01 smithi139 podman[136126]: 2024-03-24 11:31:01.216879351 +0000 UTC m=+0.140323443 container create fbcf67f78ff6787e6ef8228cf62a7a50420858ac53ba44069d3f4d49f16250b9 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6-activate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-24T11:31:01.858 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:31:01 smithi139 podman[136126]: 2024-03-24 11:31:01.580583464 +0000 UTC m=+0.504027566 container init fbcf67f78ff6787e6ef8228cf62a7a50420858ac53ba44069d3f4d49f16250b9 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6-activate, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-db0330b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-24T11:31:01.858 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:31:01 smithi139 podman[136126]: 2024-03-24 11:31:01.583880498 +0000 UTC m=+0.507324596 container start fbcf67f78ff6787e6ef8228cf62a7a50420858ac53ba44069d3f4d49f16250b9 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6-activate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, org.label-schema.build-date=20240307, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-24T11:31:01.858 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:31:01 smithi139 podman[136126]: 2024-03-24 11:31:01.694571092 +0000 UTC m=+0.618015190 container attach fbcf67f78ff6787e6ef8228cf62a7a50420858ac53ba44069d3f4d49f16250b9 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6-activate, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-24T11:31:02.358 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:31:02 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6-activate[136142]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-03-24T11:31:02.358 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:31:02 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6-activate[136142]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-6 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-24T11:31:02.358 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:31:02 smithi139 bash[136126]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-03-24T11:31:02.358 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:31:02 smithi139 bash[136126]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-6 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-24T11:31:02.359 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:31:02 smithi139 bash[136126]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-24T11:31:02.359 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:31:02 smithi139 bash[136126]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-24T11:31:02.359 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:31:02 smithi139 bash[136126]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-6/block 2024-03-24T11:31:02.359 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:31:02 smithi139 bash[136126]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-03-24T11:31:02.359 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:31:02 smithi139 bash[136126]: --> ceph-volume raw activate successful for osd ID: 6 2024-03-24T11:31:02.359 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:31:02 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6-activate[136142]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-24T11:31:02.359 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:31:02 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6-activate[136142]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-24T11:31:02.359 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:31:02 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6-activate[136142]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-6/block 2024-03-24T11:31:02.359 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:31:02 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6-activate[136142]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-03-24T11:31:02.359 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:31:02 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6-activate[136142]: --> ceph-volume raw activate successful for osd ID: 6 2024-03-24T11:31:02.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:02 smithi062 ceph-mon[90553]: pgmap v284: 161 pgs: 14 active+undersized+degraded, 9 peering, 1 stale+active+clean, 29 active+undersized, 108 active+clean; 585 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s; 58/630 objects degraded (9.206%) 2024-03-24T11:31:02.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:02 smithi062 ceph-mon[98643]: pgmap v284: 161 pgs: 14 active+undersized+degraded, 9 peering, 1 stale+active+clean, 29 active+undersized, 108 active+clean; 585 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s; 58/630 objects degraded (9.206%) 2024-03-24T11:31:02.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:02 smithi139 ceph-mon[91894]: pgmap v284: 161 pgs: 14 active+undersized+degraded, 9 peering, 1 stale+active+clean, 29 active+undersized, 108 active+clean; 585 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s; 58/630 objects degraded (9.206%) 2024-03-24T11:31:02.858 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:31:02 smithi139 podman[136126]: 2024-03-24 11:31:02.612907936 +0000 UTC m=+1.536352021 container died fbcf67f78ff6787e6ef8228cf62a7a50420858ac53ba44069d3f4d49f16250b9 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6-activate, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-db0330b, org.label-schema.build-date=20240307) 2024-03-24T11:31:03.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:03 smithi139 ceph-mon[91894]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:31:03.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:03 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:31:03.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:03 smithi062 ceph-mon[98643]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:31:04.249 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:31:03 smithi139 podman[136335]: 2024-03-24 11:31:03.989179864 +0000 UTC m=+1.900672127 container remove fbcf67f78ff6787e6ef8228cf62a7a50420858ac53ba44069d3f4d49f16250b9 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6-activate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-db0330b, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T11:31:05.034 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:04 smithi062 ceph-mon[98643]: pgmap v285: 161 pgs: 17 active+undersized+degraded, 36 active+undersized, 108 active+clean; 585 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 73/630 objects degraded (11.587%) 2024-03-24T11:31:05.034 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:05.035 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:05.035 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:04 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:05.035 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:04 smithi062 ceph-mon[90553]: pgmap v285: 161 pgs: 17 active+undersized+degraded, 36 active+undersized, 108 active+clean; 585 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 73/630 objects degraded (11.587%) 2024-03-24T11:31:05.035 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:05.035 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:05.035 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:04 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:05.108 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:31:04 smithi139 podman[136620]: 2024-03-24 11:31:04.736333521 +0000 UTC m=+0.020432603 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:31:05.108 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:31:04 smithi139 podman[136620]: 2024-03-24 11:31:04.843939157 +0000 UTC m=+0.128038230 container create c67fd9426ac5cb96eb7389e2f66c5f38f3e9c518dd9aad1e68a6a8520bc41d7d (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-db0330b, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True) 2024-03-24T11:31:05.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:04 smithi139 ceph-mon[91894]: pgmap v285: 161 pgs: 17 active+undersized+degraded, 36 active+undersized, 108 active+clean; 585 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 73/630 objects degraded (11.587%) 2024-03-24T11:31:05.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:05.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:05.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:04 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:05.936 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:31:05 smithi139 podman[136620]: 2024-03-24 11:31:05.674914936 +0000 UTC m=+0.959014010 container init c67fd9426ac5cb96eb7389e2f66c5f38f3e9c518dd9aad1e68a6a8520bc41d7d (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-db0330b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307) 2024-03-24T11:31:05.937 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:31:05 smithi139 podman[136620]: 2024-03-24 11:31:05.678031476 +0000 UTC m=+0.962130557 container start c67fd9426ac5cb96eb7389e2f66c5f38f3e9c518dd9aad1e68a6a8520bc41d7d (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-db0330b, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-24T11:31:05.937 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:31:05 smithi139 ceph-osd[136725]: -- 172.21.15.139:0/3008866885 <== mon.2 v2:172.21.15.139:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b0da80f1e0 con 0x55b0da7e1400 2024-03-24T11:31:05.937 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:31:05 smithi139 bash[136620]: c67fd9426ac5cb96eb7389e2f66c5f38f3e9c518dd9aad1e68a6a8520bc41d7d 2024-03-24T11:31:05.937 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:31:05 smithi139 systemd[1]: Started Ceph osd.6 for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:31:06.477 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:31:06 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:31:06.038+0000 7f4e4aad7700 -1 mgr.server reply reply (16) Device or resource busy unsafe to stop osd(s) at this time (19 PGs are or would become offline) 2024-03-24T11:31:06.608 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:31:06 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:31:06.211+0000 7fa78ebcf7c0 -1 Falling back to public interface 2024-03-24T11:31:07.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:06 smithi139 ceph-mon[91894]: pgmap v286: 161 pgs: 17 active+undersized+degraded, 36 active+undersized, 108 active+clean; 585 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 459 B/s rd, 0 op/s; 73/630 objects degraded (11.587%) 2024-03-24T11:31:07.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:07.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:07.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:07.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:07.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:07.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:06 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:07.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:07.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:07.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:06 smithi139 ceph-mon[91894]: pgmap v287: 161 pgs: 17 active+undersized+degraded, 36 active+undersized, 108 active+clean; 585 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 478 B/s rd, 0 op/s; 73/630 objects degraded (11.587%) 2024-03-24T11:31:07.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:07.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:07.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:07.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:07.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:07.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:06 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2024-03-24T11:31:07.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:06 smithi139 ceph-mon[91894]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2024-03-24T11:31:07.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:06 smithi139 ceph-mon[91894]: Upgrade: unsafe to stop osd(s) at this time (19 PGs are or would become offline) 2024-03-24T11:31:07.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:06 smithi062 ceph-mon[90553]: pgmap v286: 161 pgs: 17 active+undersized+degraded, 36 active+undersized, 108 active+clean; 585 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 459 B/s rd, 0 op/s; 73/630 objects degraded (11.587%) 2024-03-24T11:31:07.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:07.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:07.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:07.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:07.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:07.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:06 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:07.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:07.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:07.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:06 smithi062 ceph-mon[90553]: pgmap v287: 161 pgs: 17 active+undersized+degraded, 36 active+undersized, 108 active+clean; 585 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 478 B/s rd, 0 op/s; 73/630 objects degraded (11.587%) 2024-03-24T11:31:07.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:07.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:07.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:07.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:07.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:07.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:06 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2024-03-24T11:31:07.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:06 smithi062 ceph-mon[90553]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2024-03-24T11:31:07.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:06 smithi062 ceph-mon[90553]: Upgrade: unsafe to stop osd(s) at this time (19 PGs are or would become offline) 2024-03-24T11:31:07.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:06 smithi062 ceph-mon[98643]: pgmap v286: 161 pgs: 17 active+undersized+degraded, 36 active+undersized, 108 active+clean; 585 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 459 B/s rd, 0 op/s; 73/630 objects degraded (11.587%) 2024-03-24T11:31:07.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:07.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:07.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:07.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:07.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:07.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:06 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:07.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:07.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:07.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:06 smithi062 ceph-mon[98643]: pgmap v287: 161 pgs: 17 active+undersized+degraded, 36 active+undersized, 108 active+clean; 585 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 478 B/s rd, 0 op/s; 73/630 objects degraded (11.587%) 2024-03-24T11:31:07.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:07.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:07.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:07.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:07.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:07.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:06 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2024-03-24T11:31:07.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:06 smithi062 ceph-mon[98643]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2024-03-24T11:31:07.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:06 smithi062 ceph-mon[98643]: Upgrade: unsafe to stop osd(s) at this time (19 PGs are or would become offline) 2024-03-24T11:31:07.910 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:07 smithi062 ceph-mon[90553]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-03-24T11:31:07.910 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:07 smithi062 ceph-mon[98643]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-03-24T11:31:08.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:07 smithi139 ceph-mon[91894]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-03-24T11:31:09.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:08 smithi139 ceph-mon[91894]: pgmap v288: 161 pgs: 17 active+undersized+degraded, 36 active+undersized, 108 active+clean; 585 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 391 B/s rd, 0 op/s; 73/630 objects degraded (11.587%) 2024-03-24T11:31:09.108 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:31:08 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:31:08.959+0000 7fa78ebcf7c0 -1 osd.6 0 read_superblock omap replica is missing. 2024-03-24T11:31:09.108 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:31:08 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:31:08.987+0000 7fa78ebcf7c0 -1 osd.6 163 log_to_monitors true 2024-03-24T11:31:09.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:08 smithi062 ceph-mon[90553]: pgmap v288: 161 pgs: 17 active+undersized+degraded, 36 active+undersized, 108 active+clean; 585 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 391 B/s rd, 0 op/s; 73/630 objects degraded (11.587%) 2024-03-24T11:31:09.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:08 smithi062 ceph-mon[98643]: pgmap v288: 161 pgs: 17 active+undersized+degraded, 36 active+undersized, 108 active+clean; 585 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 391 B/s rd, 0 op/s; 73/630 objects degraded (11.587%) 2024-03-24T11:31:10.108 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:31:09 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:31:09.870+0000 7fa780027700 -1 osd.6 163 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T11:31:10.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:09 smithi139 ceph-mon[91894]: from='osd.6 [v2:172.21.15.139:6816/32485250,v1:172.21.15.139:6817/32485250]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-24T11:31:10.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:09 smithi139 ceph-mon[91894]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-24T11:31:10.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:09 smithi139 ceph-mon[91894]: Health check update: Degraded data redundancy: 73/630 objects degraded (11.587%), 17 pgs degraded (PG_DEGRADED) 2024-03-24T11:31:10.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:09 smithi062 ceph-mon[98643]: from='osd.6 [v2:172.21.15.139:6816/32485250,v1:172.21.15.139:6817/32485250]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-24T11:31:10.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:09 smithi062 ceph-mon[98643]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-24T11:31:10.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:09 smithi062 ceph-mon[98643]: Health check update: Degraded data redundancy: 73/630 objects degraded (11.587%), 17 pgs degraded (PG_DEGRADED) 2024-03-24T11:31:10.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:09 smithi062 ceph-mon[90553]: from='osd.6 [v2:172.21.15.139:6816/32485250,v1:172.21.15.139:6817/32485250]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-24T11:31:10.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:09 smithi062 ceph-mon[90553]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-24T11:31:10.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:09 smithi062 ceph-mon[90553]: Health check update: Degraded data redundancy: 73/630 objects degraded (11.587%), 17 pgs degraded (PG_DEGRADED) 2024-03-24T11:31:11.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:31:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:31:10] "GET /metrics HTTP/1.1" 200 38608 "" "Prometheus/2.33.4" 2024-03-24T11:31:11.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:10 smithi062 ceph-mon[98643]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-24T11:31:11.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:10 smithi062 ceph-mon[98643]: failed to encode map e166 with expected crc 2024-03-24T11:31:11.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:10 smithi062 ceph-mon[98643]: osdmap e166: 8 total, 7 up, 8 in 2024-03-24T11:31:11.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:10 smithi062 ceph-mon[98643]: from='osd.6 [v2:172.21.15.139:6816/32485250,v1:172.21.15.139:6817/32485250]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:31:11.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:10 smithi062 ceph-mon[98643]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:31:11.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:10 smithi062 ceph-mon[98643]: pgmap v290: 161 pgs: 17 active+undersized+degraded, 36 active+undersized, 108 active+clean; 585 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 73/630 objects degraded (11.587%) 2024-03-24T11:31:11.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:11.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:11.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:11.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:10 smithi062 ceph-mon[90553]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-24T11:31:11.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:10 smithi062 ceph-mon[90553]: failed to encode map e166 with expected crc 2024-03-24T11:31:11.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:10 smithi062 ceph-mon[90553]: osdmap e166: 8 total, 7 up, 8 in 2024-03-24T11:31:11.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:10 smithi062 ceph-mon[90553]: from='osd.6 [v2:172.21.15.139:6816/32485250,v1:172.21.15.139:6817/32485250]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:31:11.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:10 smithi062 ceph-mon[90553]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:31:11.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:10 smithi062 ceph-mon[90553]: pgmap v290: 161 pgs: 17 active+undersized+degraded, 36 active+undersized, 108 active+clean; 585 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 73/630 objects degraded (11.587%) 2024-03-24T11:31:11.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:11.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:11.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:11.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:10 smithi139 ceph-mon[91894]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-24T11:31:11.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:10 smithi139 ceph-mon[91894]: failed to encode map e166 with expected crc 2024-03-24T11:31:11.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:10 smithi139 ceph-mon[91894]: osdmap e166: 8 total, 7 up, 8 in 2024-03-24T11:31:11.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:10 smithi139 ceph-mon[91894]: from='osd.6 [v2:172.21.15.139:6816/32485250,v1:172.21.15.139:6817/32485250]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:31:11.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:10 smithi139 ceph-mon[91894]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:31:11.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:10 smithi139 ceph-mon[91894]: pgmap v290: 161 pgs: 17 active+undersized+degraded, 36 active+undersized, 108 active+clean; 585 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 73/630 objects degraded (11.587%) 2024-03-24T11:31:11.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:11.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:11.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:12.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:11 smithi062 ceph-mon[98643]: pgmap v291: 161 pgs: 17 active+undersized+degraded, 36 active+undersized, 108 active+clean; 585 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s; 73/630 objects degraded (11.587%) 2024-03-24T11:31:12.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:11 smithi062 ceph-mon[98643]: pgmap v292: 161 pgs: 17 active+undersized+degraded, 36 active+undersized, 108 active+clean; 585 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 73/630 objects degraded (11.587%) 2024-03-24T11:31:12.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:11 smithi062 ceph-mon[98643]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T11:31:12.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:11 smithi062 ceph-mon[98643]: failed to encode map e167 with expected crc 2024-03-24T11:31:12.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:11 smithi062 ceph-mon[98643]: osd.6 [v2:172.21.15.139:6816/32485250,v1:172.21.15.139:6817/32485250] boot 2024-03-24T11:31:12.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:11 smithi062 ceph-mon[98643]: osdmap e167: 8 total, 8 up, 8 in 2024-03-24T11:31:12.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T11:31:12.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:12.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:31:12.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:11 smithi062 ceph-mon[98643]: osdmap e168: 8 total, 8 up, 8 in 2024-03-24T11:31:12.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:11 smithi062 ceph-mon[90553]: pgmap v291: 161 pgs: 17 active+undersized+degraded, 36 active+undersized, 108 active+clean; 585 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s; 73/630 objects degraded (11.587%) 2024-03-24T11:31:12.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:11 smithi062 ceph-mon[90553]: pgmap v292: 161 pgs: 17 active+undersized+degraded, 36 active+undersized, 108 active+clean; 585 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 73/630 objects degraded (11.587%) 2024-03-24T11:31:12.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:11 smithi062 ceph-mon[90553]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T11:31:12.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:11 smithi062 ceph-mon[90553]: failed to encode map e167 with expected crc 2024-03-24T11:31:12.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:11 smithi062 ceph-mon[90553]: osd.6 [v2:172.21.15.139:6816/32485250,v1:172.21.15.139:6817/32485250] boot 2024-03-24T11:31:12.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:11 smithi062 ceph-mon[90553]: osdmap e167: 8 total, 8 up, 8 in 2024-03-24T11:31:12.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T11:31:12.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:12.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:31:12.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:11 smithi062 ceph-mon[90553]: osdmap e168: 8 total, 8 up, 8 in 2024-03-24T11:31:12.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:11 smithi139 ceph-mon[91894]: pgmap v291: 161 pgs: 17 active+undersized+degraded, 36 active+undersized, 108 active+clean; 585 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s; 73/630 objects degraded (11.587%) 2024-03-24T11:31:12.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:11 smithi139 ceph-mon[91894]: pgmap v292: 161 pgs: 17 active+undersized+degraded, 36 active+undersized, 108 active+clean; 585 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 73/630 objects degraded (11.587%) 2024-03-24T11:31:12.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:11 smithi139 ceph-mon[91894]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T11:31:12.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:11 smithi139 ceph-mon[91894]: failed to encode map e167 with expected crc 2024-03-24T11:31:12.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:11 smithi139 ceph-mon[91894]: osd.6 [v2:172.21.15.139:6816/32485250,v1:172.21.15.139:6817/32485250] boot 2024-03-24T11:31:12.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:11 smithi139 ceph-mon[91894]: osdmap e167: 8 total, 8 up, 8 in 2024-03-24T11:31:12.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T11:31:12.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:12.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:31:12.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:11 smithi139 ceph-mon[91894]: osdmap e168: 8 total, 8 up, 8 in 2024-03-24T11:31:13.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:12 smithi062 ceph-mon[90553]: failed to encode map e168 with expected crc 2024-03-24T11:31:13.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:12 smithi062 ceph-mon[90553]: failed to encode map e168 with expected crc 2024-03-24T11:31:13.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:12 smithi062 ceph-mon[98643]: failed to encode map e168 with expected crc 2024-03-24T11:31:13.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:12 smithi062 ceph-mon[98643]: failed to encode map e168 with expected crc 2024-03-24T11:31:13.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:12 smithi139 ceph-mon[91894]: failed to encode map e168 with expected crc 2024-03-24T11:31:13.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:12 smithi139 ceph-mon[91894]: failed to encode map e168 with expected crc 2024-03-24T11:31:14.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:13 smithi062 ceph-mon[90553]: pgmap v295: 161 pgs: 14 active+undersized+degraded, 30 active+undersized, 117 active+clean; 585 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 58/630 objects degraded (9.206%) 2024-03-24T11:31:14.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:13 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:31:14.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:13 smithi062 ceph-mon[98643]: pgmap v295: 161 pgs: 14 active+undersized+degraded, 30 active+undersized, 117 active+clean; 585 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 58/630 objects degraded (9.206%) 2024-03-24T11:31:14.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:13 smithi062 ceph-mon[98643]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:31:14.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:13 smithi139 ceph-mon[91894]: pgmap v295: 161 pgs: 14 active+undersized+degraded, 30 active+undersized, 117 active+clean; 585 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 58/630 objects degraded (9.206%) 2024-03-24T11:31:14.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:13 smithi139 ceph-mon[91894]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:31:14.772 INFO:teuthology.orchestra.run.smithi062.stdout:true 2024-03-24T11:31:15.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:14 smithi062 ceph-mon[98643]: Health check update: Degraded data redundancy: 58/630 objects degraded (9.206%), 14 pgs degraded (PG_DEGRADED) 2024-03-24T11:31:15.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:14 smithi062 ceph-mon[90553]: Health check update: Degraded data redundancy: 58/630 objects degraded (9.206%), 14 pgs degraded (PG_DEGRADED) 2024-03-24T11:31:15.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:14 smithi139 ceph-mon[91894]: Health check update: Degraded data redundancy: 58/630 objects degraded (9.206%), 14 pgs degraded (PG_DEGRADED) 2024-03-24T11:31:15.443 INFO:teuthology.orchestra.run.smithi062.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T11:31:15.443 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi062 smithi062 running 5s ago 20m - - 2024-03-24T11:31:15.443 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi139 smithi139 running 9s ago 20m - - 2024-03-24T11:31:15.443 INFO:teuthology.orchestra.run.smithi062.stdout:alertmanager.a smithi062 *:9093,9094 running (10m) 5s ago 15m 25.7M - 0.23.0 ba2b418f427c 818298d58140 2024-03-24T11:31:15.443 INFO:teuthology.orchestra.run.smithi062.stdout:grafana.a smithi139 *:3000 running (9m) 9s ago 15m 53.5M - 8.3.5 dad864ee21e9 b48167e7171e 2024-03-24T11:31:15.443 INFO:teuthology.orchestra.run.smithi062.stdout:iscsi.foo.smithi062.dggsvk smithi062 running (8m) 5s ago 14m 66.2M - 3.5 e1d6a67b021e 0a98806ec839 2024-03-24T11:31:15.444 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.x smithi139 *:8443,9283,8765 running (8m) 9s ago 19m 453M - 19.0.0-1667-gdb0330b1 4f325f8440e9 d94987b11387 2024-03-24T11:31:15.444 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.y smithi062 *:8443,9283,8765 running (9m) 5s ago 21m 535M - 19.0.0-1667-gdb0330b1 4f325f8440e9 ca1a49028d65 2024-03-24T11:31:15.444 INFO:teuthology.orchestra.run.smithi062.stdout:mon.a smithi062 running (7m) 5s ago 21m 73.0M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 f7afe6e28c88 2024-03-24T11:31:15.444 INFO:teuthology.orchestra.run.smithi062.stdout:mon.b smithi139 running (6m) 9s ago 20m 53.1M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 ca357beea4dd 2024-03-24T11:31:15.444 INFO:teuthology.orchestra.run.smithi062.stdout:mon.c smithi062 running (6m) 5s ago 20m 62.5M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 459e7449ec39 2024-03-24T11:31:15.444 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.a smithi062 *:9100 running (10m) 5s ago 16m 15.8M - 1.3.1 1dbe0e931976 79d5a02a999f 2024-03-24T11:31:15.444 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.b smithi139 *:9100 running (10m) 9s ago 16m 16.3M - 1.3.1 1dbe0e931976 8f9f4608e465 2024-03-24T11:31:15.444 INFO:teuthology.orchestra.run.smithi062.stdout:osd.0 smithi062 running (5m) 5s ago 19m 60.8M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 dfe4920b49c4 2024-03-24T11:31:15.444 INFO:teuthology.orchestra.run.smithi062.stdout:osd.1 smithi062 running (4m) 5s ago 19m 53.5M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 c443b3fe97c1 2024-03-24T11:31:15.444 INFO:teuthology.orchestra.run.smithi062.stdout:osd.2 smithi062 running (3m) 5s ago 18m 47.3M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 0eba40dc96b3 2024-03-24T11:31:15.444 INFO:teuthology.orchestra.run.smithi062.stdout:osd.3 smithi062 running (2m) 5s ago 18m 56.0M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 7501d2e4c718 2024-03-24T11:31:15.444 INFO:teuthology.orchestra.run.smithi062.stdout:osd.4 smithi139 running (80s) 9s ago 18m 51.0M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 5d0f7f05ec6d 2024-03-24T11:31:15.444 INFO:teuthology.orchestra.run.smithi062.stdout:osd.5 smithi139 running (44s) 9s ago 17m 43.4M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 4ea95bfd2127 2024-03-24T11:31:15.444 INFO:teuthology.orchestra.run.smithi062.stdout:osd.6 smithi139 unknown 9s ago 17m - 4096M 2024-03-24T11:31:15.445 INFO:teuthology.orchestra.run.smithi062.stdout:osd.7 smithi139 running (16m) 9s ago 16m 67.8M 4096M 17.2.0 e1d6a67b021e bc7fadbc1384 2024-03-24T11:31:15.445 INFO:teuthology.orchestra.run.smithi062.stdout:prometheus.a smithi139 *:9095 running (9m) 9s ago 16m 71.3M - 2.33.4 514e6a882f6e 2ed47f810d33 2024-03-24T11:31:15.445 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi062.sacpds smithi062 *:8000 running (14m) 5s ago 14m 75.3M - 17.2.0 e1d6a67b021e 89f4ac308e98 2024-03-24T11:31:15.445 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi139.yljesh smithi139 *:8000 running (14m) 9s ago 14m 74.3M - 17.2.0 e1d6a67b021e e189bdc12cae 2024-03-24T11:31:15.445 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi062.xvpxnk smithi062 *:80 running (14m) 5s ago 14m 75.1M - 17.2.0 e1d6a67b021e d1aa74adab44 2024-03-24T11:31:15.445 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi139.pbtrmp smithi139 *:80 running (14m) 9s ago 14m 78.1M - 17.2.0 e1d6a67b021e 7f8e336755d7 2024-03-24T11:31:15.848 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:31:15.848 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": { 2024-03-24T11:31:15.848 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 3 2024-03-24T11:31:15.848 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:31:15.848 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": { 2024-03-24T11:31:15.849 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-24T11:31:15.849 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:31:15.849 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": { 2024-03-24T11:31:15.849 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 1, 2024-03-24T11:31:15.849 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 7 2024-03-24T11:31:15.849 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:31:15.849 INFO:teuthology.orchestra.run.smithi062.stdout: "rgw": { 2024-03-24T11:31:15.849 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4 2024-03-24T11:31:15.849 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:31:15.849 INFO:teuthology.orchestra.run.smithi062.stdout: "overall": { 2024-03-24T11:31:15.849 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 5, 2024-03-24T11:31:15.849 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 12 2024-03-24T11:31:15.849 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-24T11:31:15.849 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:31:16.188 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:31:16.189 INFO:teuthology.orchestra.run.smithi062.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69", 2024-03-24T11:31:16.189 INFO:teuthology.orchestra.run.smithi062.stdout: "in_progress": true, 2024-03-24T11:31:16.189 INFO:teuthology.orchestra.run.smithi062.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T11:31:16.189 INFO:teuthology.orchestra.run.smithi062.stdout: "services_complete": [ 2024-03-24T11:31:16.189 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr", 2024-03-24T11:31:16.189 INFO:teuthology.orchestra.run.smithi062.stdout: "mon" 2024-03-24T11:31:16.189 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-03-24T11:31:16.189 INFO:teuthology.orchestra.run.smithi062.stdout: "progress": "11/23 daemons upgraded", 2024-03-24T11:31:16.189 INFO:teuthology.orchestra.run.smithi062.stdout: "message": "Currently upgrading osd daemons", 2024-03-24T11:31:16.189 INFO:teuthology.orchestra.run.smithi062.stdout: "is_paused": false 2024-03-24T11:31:16.189 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:31:16.190 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:15 smithi062 ceph-mon[98643]: pgmap v296: 161 pgs: 10 active+undersized+degraded, 19 active+undersized, 132 active+clean; 585 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 927 B/s rd, 0 op/s; 38/630 objects degraded (6.032%) 2024-03-24T11:31:16.190 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:15 smithi062 ceph-mon[98643]: from='client.44631 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:31:16.190 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:15 smithi062 ceph-mon[98643]: from='client.? 172.21.15.62:0/1002521861' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:16.190 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:15 smithi062 ceph-mon[90553]: pgmap v296: 161 pgs: 10 active+undersized+degraded, 19 active+undersized, 132 active+clean; 585 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 927 B/s rd, 0 op/s; 38/630 objects degraded (6.032%) 2024-03-24T11:31:16.190 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:15 smithi062 ceph-mon[90553]: from='client.44631 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:31:16.190 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:15 smithi062 ceph-mon[90553]: from='client.? 172.21.15.62:0/1002521861' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:16.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:15 smithi139 ceph-mon[91894]: pgmap v296: 161 pgs: 10 active+undersized+degraded, 19 active+undersized, 132 active+clean; 585 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 927 B/s rd, 0 op/s; 38/630 objects degraded (6.032%) 2024-03-24T11:31:16.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:15 smithi139 ceph-mon[91894]: from='client.44631 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:31:16.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:15 smithi139 ceph-mon[91894]: from='client.? 172.21.15.62:0/1002521861' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:16.591 INFO:teuthology.orchestra.run.smithi062.stdout:HEALTH_WARN 1 failed cephadm daemon(s); Degraded data redundancy: 38/630 objects degraded (6.032%), 10 pgs degraded 2024-03-24T11:31:16.591 INFO:teuthology.orchestra.run.smithi062.stdout:[WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-03-24T11:31:16.591 INFO:teuthology.orchestra.run.smithi062.stdout: daemon osd.6 on smithi139 is in unknown state 2024-03-24T11:31:16.591 INFO:teuthology.orchestra.run.smithi062.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 38/630 objects degraded (6.032%), 10 pgs degraded 2024-03-24T11:31:16.591 INFO:teuthology.orchestra.run.smithi062.stdout: pg 2.2 is active+undersized+degraded, acting [5,1] 2024-03-24T11:31:16.591 INFO:teuthology.orchestra.run.smithi062.stdout: pg 2.16 is active+undersized+degraded, acting [5,2] 2024-03-24T11:31:16.591 INFO:teuthology.orchestra.run.smithi062.stdout: pg 3.3 is active+undersized+degraded, acting [4,0] 2024-03-24T11:31:16.591 INFO:teuthology.orchestra.run.smithi062.stdout: pg 3.a is active+undersized+degraded, acting [4,1] 2024-03-24T11:31:16.591 INFO:teuthology.orchestra.run.smithi062.stdout: pg 3.c is active+undersized+degraded, acting [5,3] 2024-03-24T11:31:16.591 INFO:teuthology.orchestra.run.smithi062.stdout: pg 3.10 is active+undersized+degraded, acting [0,5] 2024-03-24T11:31:16.592 INFO:teuthology.orchestra.run.smithi062.stdout: pg 3.14 is active+undersized+degraded, acting [4,7] 2024-03-24T11:31:16.592 INFO:teuthology.orchestra.run.smithi062.stdout: pg 3.1d is active+undersized+degraded, acting [5,4] 2024-03-24T11:31:16.592 INFO:teuthology.orchestra.run.smithi062.stdout: pg 4.e is active+undersized+degraded, acting [4,1] 2024-03-24T11:31:16.592 INFO:teuthology.orchestra.run.smithi062.stdout: pg 4.1f is active+undersized+degraded, acting [5,1] 2024-03-24T11:31:17.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:16 smithi062 ceph-mon[98643]: from='client.64606 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:31:17.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:16 smithi062 ceph-mon[98643]: from='client.44643 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:31:17.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:16 smithi062 ceph-mon[98643]: from='client.? 172.21.15.62:0/3701261393' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:31:17.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:16 smithi062 ceph-mon[90553]: from='client.64606 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:31:17.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:16 smithi062 ceph-mon[90553]: from='client.44643 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:31:17.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:16 smithi062 ceph-mon[90553]: from='client.? 172.21.15.62:0/3701261393' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:31:17.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:16 smithi139 ceph-mon[91894]: from='client.64606 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:31:17.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:16 smithi139 ceph-mon[91894]: from='client.44643 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:31:17.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:16 smithi139 ceph-mon[91894]: from='client.? 172.21.15.62:0/3701261393' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:31:18.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:17 smithi062 ceph-mon[90553]: from='client.44655 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:31:18.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:17 smithi062 ceph-mon[90553]: pgmap v297: 161 pgs: 161 active+clean; 585 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-03-24T11:31:18.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:17 smithi062 ceph-mon[90553]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 38/630 objects degraded (6.032%), 10 pgs degraded) 2024-03-24T11:31:18.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:17 smithi062 ceph-mon[98643]: from='client.44655 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:31:18.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:17 smithi062 ceph-mon[98643]: pgmap v297: 161 pgs: 161 active+clean; 585 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-03-24T11:31:18.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:17 smithi062 ceph-mon[98643]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 38/630 objects degraded (6.032%), 10 pgs degraded) 2024-03-24T11:31:18.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:17 smithi139 ceph-mon[91894]: from='client.44655 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:31:18.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:17 smithi139 ceph-mon[91894]: pgmap v297: 161 pgs: 161 active+clean; 585 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-03-24T11:31:18.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:17 smithi139 ceph-mon[91894]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 38/630 objects degraded (6.032%), 10 pgs degraded) 2024-03-24T11:31:20.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:19 smithi062 ceph-mon[98643]: pgmap v298: 161 pgs: 161 active+clean; 585 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-03-24T11:31:20.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:19 smithi062 ceph-mon[90553]: pgmap v298: 161 pgs: 161 active+clean; 585 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-03-24T11:31:20.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:19 smithi139 ceph-mon[91894]: pgmap v298: 161 pgs: 161 active+clean; 585 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-03-24T11:31:21.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:31:20 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:31:20] "GET /metrics HTTP/1.1" 200 38470 "" "Prometheus/2.33.4" 2024-03-24T11:31:22.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:21 smithi062 ceph-mon[90553]: pgmap v299: 161 pgs: 161 active+clean; 585 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 966 B/s rd, 0 op/s 2024-03-24T11:31:22.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2024-03-24T11:31:22.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:21 smithi062 ceph-mon[90553]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2024-03-24T11:31:22.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:21 smithi062 ceph-mon[90553]: Upgrade: osd.7 is safe to restart 2024-03-24T11:31:22.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:22.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:21 smithi062 ceph-mon[98643]: pgmap v299: 161 pgs: 161 active+clean; 585 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 966 B/s rd, 0 op/s 2024-03-24T11:31:22.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2024-03-24T11:31:22.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:21 smithi062 ceph-mon[98643]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2024-03-24T11:31:22.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:21 smithi062 ceph-mon[98643]: Upgrade: osd.7 is safe to restart 2024-03-24T11:31:22.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:22.239 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:21 smithi139 ceph-mon[91894]: pgmap v299: 161 pgs: 161 active+clean; 585 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 966 B/s rd, 0 op/s 2024-03-24T11:31:22.240 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2024-03-24T11:31:22.240 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:21 smithi139 ceph-mon[91894]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2024-03-24T11:31:22.240 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:21 smithi139 ceph-mon[91894]: Upgrade: osd.7 is safe to restart 2024-03-24T11:31:22.240 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:24.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:23 smithi062 ceph-mon[90553]: pgmap v300: 161 pgs: 161 active+clean; 585 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 874 B/s rd, 0 op/s 2024-03-24T11:31:24.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:23 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:31:24.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:24.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-24T11:31:24.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:24.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:23 smithi062 ceph-mon[98643]: pgmap v300: 161 pgs: 161 active+clean; 585 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 874 B/s rd, 0 op/s 2024-03-24T11:31:24.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:23 smithi062 ceph-mon[98643]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:31:24.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:24.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-24T11:31:24.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:24.336 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:23 smithi139 ceph-mon[91894]: pgmap v300: 161 pgs: 161 active+clean; 585 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 874 B/s rd, 0 op/s 2024-03-24T11:31:24.336 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:23 smithi139 ceph-mon[91894]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:31:24.336 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:24.336 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-24T11:31:24.336 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:25.185 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:24 smithi139 ceph-mon[91894]: Upgrade: Updating osd.7 2024-03-24T11:31:25.185 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:24 smithi139 ceph-mon[91894]: Deploying daemon osd.7 on smithi139 2024-03-24T11:31:25.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:24 smithi062 ceph-mon[90553]: Upgrade: Updating osd.7 2024-03-24T11:31:25.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:24 smithi062 ceph-mon[90553]: Deploying daemon osd.7 on smithi139 2024-03-24T11:31:25.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:24 smithi062 ceph-mon[98643]: Upgrade: Updating osd.7 2024-03-24T11:31:25.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:24 smithi062 ceph-mon[98643]: Deploying daemon osd.7 on smithi139 2024-03-24T11:31:26.041 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:25 smithi062 ceph-mon[90553]: pgmap v301: 161 pgs: 161 active+clean; 585 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:31:26.041 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:25 smithi062 ceph-mon[98643]: pgmap v301: 161 pgs: 161 active+clean; 585 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:31:26.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:25 smithi139 ceph-mon[91894]: pgmap v301: 161 pgs: 161 active+clean; 585 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:31:27.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:27.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:27.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:27 smithi062 ceph-mon[90553]: pgmap v302: 161 pgs: 161 active+clean; 585 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 873 B/s rd, 0 op/s 2024-03-24T11:31:27.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:27 smithi062 ceph-mon[90553]: pgmap v303: 161 pgs: 161 active+clean; 585 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-24T11:31:27.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:27.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:27.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:31:27.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:27.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:27.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:27 smithi062 ceph-mon[98643]: pgmap v302: 161 pgs: 161 active+clean; 585 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 873 B/s rd, 0 op/s 2024-03-24T11:31:27.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:27 smithi062 ceph-mon[98643]: pgmap v303: 161 pgs: 161 active+clean; 585 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-24T11:31:27.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:27.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:27.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:31:27.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:27.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:27.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:27 smithi139 ceph-mon[91894]: pgmap v302: 161 pgs: 161 active+clean; 585 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 873 B/s rd, 0 op/s 2024-03-24T11:31:27.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:27 smithi139 ceph-mon[91894]: pgmap v303: 161 pgs: 161 active+clean; 585 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-24T11:31:27.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:27.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:27.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:31:28.512 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:28 smithi139 systemd[1]: Stopping Ceph osd.7 for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:31:29.157 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:28 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[39163]: 2024-03-24T11:31:28.898+0000 7f200eeec700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T11:31:29.157 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:28 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[39163]: 2024-03-24T11:31:28.898+0000 7f200eeec700 -1 osd.7 168 *** Got signal Terminated *** 2024-03-24T11:31:29.158 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:28 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[39163]: 2024-03-24T11:31:28.898+0000 7f200eeec700 -1 osd.7 168 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T11:31:29.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:29 smithi062 ceph-mon[90553]: pgmap v304: 161 pgs: 161 active+clean; 585 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-24T11:31:29.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:29 smithi062 ceph-mon[90553]: osd.7 marked itself down and dead 2024-03-24T11:31:29.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:29 smithi062 ceph-mon[98643]: pgmap v304: 161 pgs: 161 active+clean; 585 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-24T11:31:29.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:29 smithi062 ceph-mon[98643]: osd.7 marked itself down and dead 2024-03-24T11:31:29.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:29 smithi139 ceph-mon[91894]: pgmap v304: 161 pgs: 161 active+clean; 585 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-24T11:31:29.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:29 smithi139 ceph-mon[91894]: osd.7 marked itself down and dead 2024-03-24T11:31:29.971 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:29 smithi139 podman[138290]: 2024-03-24 11:31:29.704530981 +0000 UTC m=+0.917904801 container died bc7fadbc1384c6a5e22ddfcc060d4dc01362ef5803117a7cc50fdf4b12e4fda5 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, architecture=x86_64, RELEASE=HEAD, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, io.openshift.tags=base centos centos-stream, io.k8s.display-name=CentOS Stream 8, distribution-scope=public, vendor=Red Hat, Inc., name=centos-stream, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, GIT_REPO=https://github.com/ceph/ceph-container.git, maintainer=Guillaume Abrioux , description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, GIT_BRANCH=HEAD, io.openshift.expose-services=, io.buildah.version=1.19.8, version=8, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, release=754, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., build-date=2022-05-03T08:36:31.336870, GIT_CLEAN=True, com.redhat.component=centos-stream-container, CEPH_POINT_RELEASE=-17.2.0, ceph=True, vcs-type=git) 2024-03-24T11:31:30.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:30 smithi139 podman[138290]: 2024-03-24 11:31:30.061106777 +0000 UTC m=+1.274480573 container cleanup bc7fadbc1384c6a5e22ddfcc060d4dc01362ef5803117a7cc50fdf4b12e4fda5 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7, com.redhat.component=centos-stream-container, io.k8s.display-name=CentOS Stream 8, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, architecture=x86_64, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, io.openshift.expose-services=, build-date=2022-05-03T08:36:31.336870, io.openshift.tags=base centos centos-stream, distribution-scope=public, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, CEPH_POINT_RELEASE=-17.2.0, release=754, vendor=Red Hat, Inc., description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, io.buildah.version=1.19.8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, version=8, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, GIT_CLEAN=True, ceph=True, vcs-type=git, name=centos-stream, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, maintainer=Guillaume Abrioux , io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., GIT_BRANCH=HEAD) 2024-03-24T11:31:30.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:30 smithi139 bash[138290]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7 2024-03-24T11:31:30.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:30 smithi139 ceph-mon[91894]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T11:31:30.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:30 smithi139 ceph-mon[91894]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-03-24T11:31:30.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:30 smithi139 ceph-mon[91894]: osdmap e169: 8 total, 7 up, 8 in 2024-03-24T11:31:30.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:30.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:30.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:30 smithi139 ceph-mon[91894]: pgmap v306: 161 pgs: 23 stale+active+clean, 138 active+clean; 585 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-03-24T11:31:30.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:30.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:30 smithi062 ceph-mon[90553]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T11:31:30.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:30 smithi062 ceph-mon[90553]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-03-24T11:31:30.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:30 smithi062 ceph-mon[90553]: osdmap e169: 8 total, 7 up, 8 in 2024-03-24T11:31:30.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:30.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:30.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:30 smithi062 ceph-mon[90553]: pgmap v306: 161 pgs: 23 stale+active+clean, 138 active+clean; 585 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-03-24T11:31:30.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:30.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:30 smithi062 ceph-mon[98643]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-24T11:31:30.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:30 smithi062 ceph-mon[98643]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-03-24T11:31:30.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:30 smithi062 ceph-mon[98643]: osdmap e169: 8 total, 7 up, 8 in 2024-03-24T11:31:30.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:30.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:30.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:30 smithi062 ceph-mon[98643]: pgmap v306: 161 pgs: 23 stale+active+clean, 138 active+clean; 585 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-03-24T11:31:30.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:30.739 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:30 smithi139 podman[138413]: 2024-03-24 11:31:30.395840911 +0000 UTC m=+0.696203668 container remove bc7fadbc1384c6a5e22ddfcc060d4dc01362ef5803117a7cc50fdf4b12e4fda5 (image=quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7, name=centos-stream, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.6-754, vcs-type=git, version=8, com.redhat.component=centos-stream-container, GIT_REPO=https://github.com/ceph/ceph-container.git, description=CentOS Stream is a continuously delivered distro that tracks just ahead of Red Hat Enterprise Linux development. This image takes the Red Hat UBI and layers on content from CentOS Stream, release=754, ceph=True, io.k8s.display-name=CentOS Stream 8, com.redhat.license_terms=https://centos.org/legal/licensing-policy/, summary=Provides a CentOS Stream container based on the Red Hat Universal Base Image, GIT_BRANCH=HEAD, io.openshift.tags=base centos centos-stream, maintainer=Guillaume Abrioux , io.openshift.expose-services=, vcs-ref=f1ee6e37554363ec55e0035aba1a693d3627fdeb, CEPH_POINT_RELEASE=-17.2.0, GIT_COMMIT=b613db0f44179c0940781c1c7fe04e1acb7093ac, architecture=x86_64, GIT_CLEAN=True, RELEASE=HEAD, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, io.buildah.version=1.19.8, vendor=Red Hat, Inc., build-date=2022-05-03T08:36:31.336870, distribution-scope=public, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly.) 2024-03-24T11:31:31.042 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:30 smithi139 podman[138503]: 2024-03-24 11:31:30.641606352 +0000 UTC m=+0.017586978 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:31:31.042 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:30 smithi139 podman[138503]: 2024-03-24 11:31:30.838842611 +0000 UTC m=+0.214823237 container create d407bda211a4a617eee282a0bfbe689aeb515ef91eab4062228c7adaec55d17f (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-24T11:31:31.042 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:31 smithi139 podman[138503]: 2024-03-24 11:31:31.042293234 +0000 UTC m=+0.418273861 container init d407bda211a4a617eee282a0bfbe689aeb515ef91eab4062228c7adaec55d17f (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7-deactivate, org.label-schema.license=GPLv2, RELEASE=squid-db0330b, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-24T11:31:31.176 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:31:30 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:31:30] "GET /metrics HTTP/1.1" 200 38615 "" "Prometheus/2.33.4" 2024-03-24T11:31:31.336 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:31 smithi139 ceph-mon[91894]: osdmap e170: 8 total, 7 up, 8 in 2024-03-24T11:31:31.336 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:31.336 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:31.336 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:31.336 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:31 smithi139 ceph-mon[91894]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-03-24T11:31:31.336 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:31 smithi139 podman[138503]: 2024-03-24 11:31:31.045733873 +0000 UTC m=+0.421714499 container start d407bda211a4a617eee282a0bfbe689aeb515ef91eab4062228c7adaec55d17f (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240307, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-24T11:31:31.336 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:31 smithi139 podman[138503]: 2024-03-24 11:31:31.124794819 +0000 UTC m=+0.500775445 container attach d407bda211a4a617eee282a0bfbe689aeb515ef91eab4062228c7adaec55d17f (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , RELEASE=squid-db0330b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-24T11:31:31.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:31 smithi062 ceph-mon[90553]: osdmap e170: 8 total, 7 up, 8 in 2024-03-24T11:31:31.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:31.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:31.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:31.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:31 smithi062 ceph-mon[90553]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-03-24T11:31:31.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:31 smithi062 ceph-mon[98643]: osdmap e170: 8 total, 7 up, 8 in 2024-03-24T11:31:31.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:31.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:31.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:31.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:31 smithi062 ceph-mon[98643]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-03-24T11:31:31.608 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:31 smithi139 podman[138503]: 2024-03-24 11:31:31.336100476 +0000 UTC m=+0.712081116 container died d407bda211a4a617eee282a0bfbe689aeb515ef91eab4062228c7adaec55d17f (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7-deactivate, RELEASE=squid-db0330b, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T11:31:32.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:32 smithi139 ceph-mon[91894]: pgmap v308: 161 pgs: 7 active+undersized+degraded, 21 peering, 5 stale+active+clean, 10 active+undersized, 118 active+clean; 585 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 17/630 objects degraded (2.698%) 2024-03-24T11:31:32.359 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:31 smithi139 podman[138529]: 2024-03-24 11:31:31.945117259 +0000 UTC m=+0.603959818 container remove d407bda211a4a617eee282a0bfbe689aeb515ef91eab4062228c7adaec55d17f (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7-deactivate, ceph=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-db0330b) 2024-03-24T11:31:32.359 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:31 smithi139 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.7.service: Deactivated successfully. 2024-03-24T11:31:32.359 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:31 smithi139 systemd[1]: Stopped Ceph osd.7 for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:31:32.359 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:31 smithi139 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.7.service: Consumed 12.737s CPU time. 2024-03-24T11:31:32.359 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:32 smithi139 systemd[1]: Starting Ceph osd.7 for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:31:32.474 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:32 smithi062 ceph-mon[90553]: pgmap v308: 161 pgs: 7 active+undersized+degraded, 21 peering, 5 stale+active+clean, 10 active+undersized, 118 active+clean; 585 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 17/630 objects degraded (2.698%) 2024-03-24T11:31:32.475 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:32 smithi062 ceph-mon[98643]: pgmap v308: 161 pgs: 7 active+undersized+degraded, 21 peering, 5 stale+active+clean, 10 active+undersized, 118 active+clean; 585 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 17/630 objects degraded (2.698%) 2024-03-24T11:31:32.765 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:32 smithi139 podman[138633]: 2024-03-24 11:31:32.289020803 +0000 UTC m=+0.017882632 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:31:32.765 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:32 smithi139 podman[138633]: 2024-03-24 11:31:32.387695819 +0000 UTC m=+0.116557643 container create 0f84da785fe0c1745b2381444720f2f7ba32454ff12b0bf06846d3c66f926ec1 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7-activate, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T11:31:32.765 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:32 smithi139 podman[138633]: 2024-03-24 11:31:32.607034203 +0000 UTC m=+0.335896034 container init 0f84da785fe0c1745b2381444720f2f7ba32454ff12b0bf06846d3c66f926ec1 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7-activate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T11:31:32.765 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:32 smithi139 podman[138633]: 2024-03-24 11:31:32.610845243 +0000 UTC m=+0.339707074 container start 0f84da785fe0c1745b2381444720f2f7ba32454ff12b0bf06846d3c66f926ec1 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307) 2024-03-24T11:31:32.765 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:32 smithi139 podman[138633]: 2024-03-24 11:31:32.765445102 +0000 UTC m=+0.494306925 container attach 0f84da785fe0c1745b2381444720f2f7ba32454ff12b0bf06846d3c66f926ec1 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7-activate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-24T11:31:33.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:33 smithi139 ceph-mon[91894]: Health check failed: Reduced data availability: 1 pg inactive, 5 pgs peering (PG_AVAILABILITY) 2024-03-24T11:31:33.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:33 smithi139 ceph-mon[91894]: Health check failed: Degraded data redundancy: 17/630 objects degraded (2.698%), 7 pgs degraded (PG_DEGRADED) 2024-03-24T11:31:33.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:33 smithi139 ceph-mon[91894]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:31:33.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:33 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7-activate[138647]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-03-24T11:31:33.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:33 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7-activate[138647]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-7 --no-mon-config --dev /dev/mapper/vg_nvme-lv_1 2024-03-24T11:31:33.359 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:33 smithi139 bash[138633]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-03-24T11:31:33.359 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:33 smithi139 bash[138633]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-7 --no-mon-config --dev /dev/mapper/vg_nvme-lv_1 2024-03-24T11:31:33.359 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:33 smithi139 bash[138633]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_1 2024-03-24T11:31:33.359 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:33 smithi139 bash[138633]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-0 2024-03-24T11:31:33.359 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:33 smithi139 bash[138633]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-7/block 2024-03-24T11:31:33.359 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:33 smithi139 bash[138633]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-03-24T11:31:33.359 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:33 smithi139 bash[138633]: --> ceph-volume raw activate successful for osd ID: 7 2024-03-24T11:31:33.359 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:33 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7-activate[138647]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_1 2024-03-24T11:31:33.359 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:33 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7-activate[138647]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-0 2024-03-24T11:31:33.359 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:33 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7-activate[138647]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-7/block 2024-03-24T11:31:33.359 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:33 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7-activate[138647]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-03-24T11:31:33.359 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:33 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7-activate[138647]: --> ceph-volume raw activate successful for osd ID: 7 2024-03-24T11:31:33.359 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:33 smithi139 podman[138633]: 2024-03-24 11:31:33.080449903 +0000 UTC m=+0.809311725 container died 0f84da785fe0c1745b2381444720f2f7ba32454ff12b0bf06846d3c66f926ec1 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7-activate, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-db0330b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-24T11:31:33.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:33 smithi062 ceph-mon[98643]: Health check failed: Reduced data availability: 1 pg inactive, 5 pgs peering (PG_AVAILABILITY) 2024-03-24T11:31:33.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:33 smithi062 ceph-mon[98643]: Health check failed: Degraded data redundancy: 17/630 objects degraded (2.698%), 7 pgs degraded (PG_DEGRADED) 2024-03-24T11:31:33.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:33 smithi062 ceph-mon[98643]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:31:33.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:33 smithi062 ceph-mon[90553]: Health check failed: Reduced data availability: 1 pg inactive, 5 pgs peering (PG_AVAILABILITY) 2024-03-24T11:31:33.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:33 smithi062 ceph-mon[90553]: Health check failed: Degraded data redundancy: 17/630 objects degraded (2.698%), 7 pgs degraded (PG_DEGRADED) 2024-03-24T11:31:33.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:33 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:31:34.108 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:33 smithi139 podman[138780]: 2024-03-24 11:31:33.786216879 +0000 UTC m=+0.700546463 container remove 0f84da785fe0c1745b2381444720f2f7ba32454ff12b0bf06846d3c66f926ec1 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7-activate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-db0330b, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T11:31:34.108 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:34 smithi139 podman[138841]: 2024-03-24 11:31:33.909795925 +0000 UTC m=+0.018705557 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:31:34.108 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:34 smithi139 podman[138841]: 2024-03-24 11:31:34.012193005 +0000 UTC m=+0.121102632 container create d4fbd9bccd09fdf1dbef693bb02c64a92d96f60b75188faf96c0edef5a4f4cf3 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7, GIT_CLEAN=True, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-db0330b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T11:31:34.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:34 smithi062 ceph-mon[90553]: pgmap v309: 161 pgs: 11 active+undersized+degraded, 21 peering, 4 stale+active+clean, 11 active+undersized, 114 active+clean; 585 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 24/630 objects degraded (3.810%) 2024-03-24T11:31:34.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:34 smithi062 ceph-mon[98643]: pgmap v309: 161 pgs: 11 active+undersized+degraded, 21 peering, 4 stale+active+clean, 11 active+undersized, 114 active+clean; 585 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 24/630 objects degraded (3.810%) 2024-03-24T11:31:34.566 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:34 smithi139 ceph-mon[91894]: pgmap v309: 161 pgs: 11 active+undersized+degraded, 21 peering, 4 stale+active+clean, 11 active+undersized, 114 active+clean; 585 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 24/630 objects degraded (3.810%) 2024-03-24T11:31:34.566 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:34 smithi139 podman[138841]: 2024-03-24 11:31:34.273258147 +0000 UTC m=+0.382167775 container init d4fbd9bccd09fdf1dbef693bb02c64a92d96f60b75188faf96c0edef5a4f4cf3 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux ) 2024-03-24T11:31:34.566 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:34 smithi139 podman[138841]: 2024-03-24 11:31:34.276360638 +0000 UTC m=+0.385270269 container start d4fbd9bccd09fdf1dbef693bb02c64a92d96f60b75188faf96c0edef5a4f4cf3 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7, RELEASE=squid-db0330b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-24T11:31:34.566 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:34 smithi139 ceph-osd[138859]: -- 172.21.15.139:0/1017238203 <== mon.2 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557fd7be7600 con 0x557fd7b6ec00 2024-03-24T11:31:34.567 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:34 smithi139 ceph-osd[138859]: -- 172.21.15.139:0/1017238203 <== mon.2 v2:172.21.15.139:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557fd7b9d1e0 con 0x557fd7b6ec00 2024-03-24T11:31:34.567 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:34 smithi139 bash[138841]: d4fbd9bccd09fdf1dbef693bb02c64a92d96f60b75188faf96c0edef5a4f4cf3 2024-03-24T11:31:34.567 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:34 smithi139 systemd[1]: Started Ceph osd.7 for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:31:35.108 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:34 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:31:34.816+0000 7fe4199b07c0 -1 Falling back to public interface 2024-03-24T11:31:35.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:35.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:35.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:35.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:35.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:35.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:35.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:35.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:35.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:35.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:35.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:35.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:35.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:35.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:35.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:35.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:35.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:35.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:35.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:35.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:35.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:35.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:35.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:35.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:35.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:35.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:35.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:35.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:35.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:35.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:35.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:35.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:35.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:35.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:35.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:35.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:35.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:35.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:35.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:35.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:35.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:35.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:35.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:35.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:35.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:35.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:35.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:35.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:35.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:35.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:35.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:35.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:35.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:35.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:35.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:35.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:35.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:35.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:35.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:35.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:35.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:35.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:35.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:35.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:35.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:35.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:35.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:35.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:35.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:35.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:35.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:35.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:35.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:35.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:35.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:35.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:35.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:35.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:35.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:35.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:35.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:35.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:35.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:35.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:35.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:35.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:35.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:35.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:35.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:35.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:35.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:35.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:35.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:35.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:35.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:35.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:35.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:35.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:35.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:35.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:35.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:35.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:35.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:35.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:35.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:35.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:35 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:35.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:35.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:35.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:35.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:35.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:35.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:35 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:35.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:35.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:35.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:35.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:35.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:35.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:35 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:35.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:35.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:35.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:35.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:35.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:35.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:35 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:35.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:35.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:35.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:35.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:35.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:35.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:35 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:35.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:35.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:35.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:35.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:35.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:35.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:35.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:35.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:35.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:35.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:35.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:35.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:35.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:35.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:35.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:35.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:35.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:35.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:35.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:35.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:36.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:36.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:36.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:36.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:36.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:36.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:36.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:36.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:36.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:36.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:36.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:36.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:36.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:36.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:36.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:36.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:36.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:36.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:36.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:36.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:36.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:36.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:36.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:36.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:36.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:36.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: pgmap v310: 161 pgs: 27 active+undersized+degraded, 42 active+undersized, 92 active+clean; 585 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 85/630 objects degraded (13.492%) 2024-03-24T11:31:36.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:36.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:36.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:36.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:36.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:36.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:36.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:36.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:36.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:36.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:36.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:36.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:36.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:36.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:36.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:36.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:36.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:36.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:36.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:36.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:36.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:36.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:36.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:36.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:36.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:36.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:36.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:36.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:36.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:36.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:36.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:36.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:36.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:36 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:36.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:36.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:36.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:36.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:36.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:36.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:36.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:36.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:36.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:36.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:36.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:36.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:36.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:36.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:36.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:36.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:36.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:36.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:36.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:36.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:36.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:36.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:36.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:36.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:36.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:36.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: pgmap v310: 161 pgs: 27 active+undersized+degraded, 42 active+undersized, 92 active+clean; 585 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 85/630 objects degraded (13.492%) 2024-03-24T11:31:36.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:36.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:36.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:36.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:36.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:36.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:36.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:36.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:36.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:36.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:36.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:36.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:36.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:36.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:36.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:36.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:36.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:36.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:36.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:36.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:36.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:36.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:36.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:36.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:36.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:36.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:36.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:36.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:36.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: pgmap v310: 161 pgs: 27 active+undersized+degraded, 42 active+undersized, 92 active+clean; 585 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 85/630 objects degraded (13.492%) 2024-03-24T11:31:36.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:36.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:36.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:36.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:36.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:36.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:36.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:36.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:36.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:36.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:36.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:36.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:36.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:36.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:36.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:36.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:36.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:36.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:36.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:36.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:36.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:36.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:36.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:36.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:36.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:36.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:36.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:36.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:36.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:36.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:36.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:36.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:36.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:36.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:36.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:36.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:36.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:36.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:36.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:36.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:36.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:36.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:36.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:36.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:36.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:36.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:36.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:36.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:36.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:36.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:36.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:36.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:36.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:36.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:36.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:36.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:36.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:36.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:36.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:36.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:36.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:36.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:36.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:36 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:37.694 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:37 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:31:37.576+0000 7fe4199b07c0 -1 osd.7 0 read_superblock omap replica is missing. 2024-03-24T11:31:37.695 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:37 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:31:37.625+0000 7fe4199b07c0 -1 osd.7 168 log_to_monitors true 2024-03-24T11:31:37.695 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:37 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:37.695 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:37 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:37.695 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:37 smithi139 ceph-mon[91894]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 5 pgs peering) 2024-03-24T11:31:37.695 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:37.695 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:37.695 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:37.695 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:37.695 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:37.696 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:37 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:37.696 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:37.696 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:37.696 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:37.696 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:37.696 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:37.696 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:37.696 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:37.696 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:37.696 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:37.696 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:37.696 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:37.696 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:37.696 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:37.697 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:37.697 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:37.697 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:37.697 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:37.697 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:37.697 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:37.697 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:37 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:37.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:37.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:37.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[98643]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 5 pgs peering) 2024-03-24T11:31:37.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:37.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:37.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:37.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:37.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:37.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:37.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:37.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:37.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:37.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:37.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:37.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:37.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:37.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:37.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:37.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:37.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:37.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:37.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:37.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:37.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:37.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:37.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:37.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:37.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:37.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:37.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:37.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:37.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[90553]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 5 pgs peering) 2024-03-24T11:31:37.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:37.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:37.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:37.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:37.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:37.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:37.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:37.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:37.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:37.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:37.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:37.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:37.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:37.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:37.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:37.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:37.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:37.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:37.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:37.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:37.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:37.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:37.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:37.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:37.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:37.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:37 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:38.701 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:31:38.553+0000 7fe40ae08700 -1 osd.7 168 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T11:31:38.702 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:38.702 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:38.702 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:38.702 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:38.702 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:38.702 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:38.702 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:38.703 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:38.703 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:38.703 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:38.703 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:38.703 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:38.703 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:38.703 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:38.703 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:38.703 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:38.703 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='osd.7 [v2:172.21.15.139:6824/2848559150,v1:172.21.15.139:6825/2848559150]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-24T11:31:38.703 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-24T11:31:38.703 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:38.703 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:38.703 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:38.703 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:38.703 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:38.704 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:38.704 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:38.704 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:38.704 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:38.704 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:38.704 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:38.704 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:38.704 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: pgmap v311: 161 pgs: 27 active+undersized+degraded, 42 active+undersized, 92 active+clean; 585 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 85/630 objects degraded (13.492%) 2024-03-24T11:31:38.705 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:38.705 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:38.705 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:38.705 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:38.705 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:38.705 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:38.705 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:38.705 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:38.705 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:38.705 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:38.705 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:38.705 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:38.705 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:38.705 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:38.705 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:38.706 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:38.706 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:38.706 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:38.706 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:38.706 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:38.706 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:38.706 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:38.706 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:38.706 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:38.706 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:38.706 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:38.706 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:38.706 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:38.706 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:38.707 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:38.707 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:38.707 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:38.707 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:38.707 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:38.707 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:38.707 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:38.707 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:38.707 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:38.707 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:38.707 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:38.707 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:38.707 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:38.707 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:38 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:38.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:38.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:38.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:38.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:38.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:38.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:38.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:38.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:38.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:38.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:38.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:38.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:38.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:38.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:38.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:38.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:38.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='osd.7 [v2:172.21.15.139:6824/2848559150,v1:172.21.15.139:6825/2848559150]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-24T11:31:38.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-24T11:31:38.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:38.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:38.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:38.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:38.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:38.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:38.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:38.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:38.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:38.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:38.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:38.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:38.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:38.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:38.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:38.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:38.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:38.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:38.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:38.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:38.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:38.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:38.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:38.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:38.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:38.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:38.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:38.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='osd.7 [v2:172.21.15.139:6824/2848559150,v1:172.21.15.139:6825/2848559150]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-24T11:31:38.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-24T11:31:38.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:38.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:38.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:38.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:38.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:38.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:38.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:38.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:38.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:38.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:38.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:38.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:38.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: pgmap v311: 161 pgs: 27 active+undersized+degraded, 42 active+undersized, 92 active+clean; 585 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 85/630 objects degraded (13.492%) 2024-03-24T11:31:38.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:38.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:38.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:38.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:38.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:38.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:38.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:38.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:38.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:38.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:38.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:38.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:38.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:38.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:38.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:38.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:38.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:38.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:38.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:38.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:38.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:38.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:38.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:38.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:38.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:38.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:38.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:38.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:38.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:38.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:38.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:38.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:38.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:38.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:38.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:38.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:38.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:38.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:38.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:38.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:38.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:38.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:38.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:38.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:38.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: pgmap v311: 161 pgs: 27 active+undersized+degraded, 42 active+undersized, 92 active+clean; 585 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 85/630 objects degraded (13.492%) 2024-03-24T11:31:38.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:38.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:38.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:38.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:38.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:38.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:38.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:38.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:38.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:38.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:38.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:38.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:38.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:38.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:38.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:38.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:38.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:38.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:38.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:38.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:38.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:38.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:38.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:38.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:38.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:38.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:38.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:38.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:38.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:38.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:38.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:38.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:38.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:38.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:38.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:38.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:38.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:38.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:38.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:38.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:38.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:38.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:38.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:38 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:39.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:39.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:39.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:39.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:39.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:39.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-24T11:31:39.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: osdmap e171: 8 total, 7 up, 8 in 2024-03-24T11:31:39.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='osd.7 [v2:172.21.15.139:6824/2848559150,v1:172.21.15.139:6825/2848559150]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:31:39.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:31:39.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:39.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:39.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:39.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:39.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:39.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:39.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:39.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:39.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:39.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:39.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:39.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:39.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:39.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:39.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:39.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:39.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:39.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:39.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:39.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:39.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:39.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:39.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:39.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:39.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:39.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:39.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:39.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:39.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:39.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:39.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:39.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:39.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:39.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:39.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:39.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:39.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:39.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:39.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:39.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:39.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:39.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:39.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:39.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:39.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:39.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:39.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:39.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:39.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:39.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:39.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:39.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:39.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:39.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:39.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-24T11:31:39.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: osdmap e171: 8 total, 7 up, 8 in 2024-03-24T11:31:39.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='osd.7 [v2:172.21.15.139:6824/2848559150,v1:172.21.15.139:6825/2848559150]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:31:39.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:31:39.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:39.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:39.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:39.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:39.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:39.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:39.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:39.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:39.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:39.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:39.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:39.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:39.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:39.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:39.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:39.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:39.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:39.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:39.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:39.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:39.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:39.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:39.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: Health check update: Degraded data redundancy: 85/630 objects degraded (13.492%), 27 pgs degraded (PG_DEGRADED) 2024-03-24T11:31:39.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:39.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:39.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:39.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:39.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:39.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:39.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:39.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:39.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:39.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:39.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:39.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:39.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:39.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:39.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:39.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:39.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:39.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:39.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:39.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:39.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:39.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:39.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:39.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:39.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:39.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:39.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:39.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:39.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:39.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:39.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:39.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:39.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:39.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:39.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:39.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:39.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:39.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:39.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:39.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:39.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:39.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: Health check update: Degraded data redundancy: 85/630 objects degraded (13.492%), 27 pgs degraded (PG_DEGRADED) 2024-03-24T11:31:39.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:39.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:39.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:39.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:39.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:39.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:39.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:39.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:39.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:39.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:39 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:39.804 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:39.804 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:39.804 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:39.805 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:39.805 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:39.805 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-24T11:31:39.805 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: osdmap e171: 8 total, 7 up, 8 in 2024-03-24T11:31:39.805 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='osd.7 [v2:172.21.15.139:6824/2848559150,v1:172.21.15.139:6825/2848559150]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:31:39.805 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-03-24T11:31:39.805 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:39.805 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:39.805 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:39.805 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:39.805 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:39.805 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:39.805 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:39.805 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:39.806 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:39.806 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:39.806 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:39.806 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:39.806 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:39.806 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:39.806 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:39.806 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:39.806 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:39.806 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:39.806 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:39.806 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:39.807 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:39.807 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:39.807 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:39.807 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:39.807 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:39.807 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:39.807 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:39.807 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:39.807 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:39.807 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:39.807 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:39.808 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:39.808 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:39.808 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:39.808 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:39.808 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:39.808 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:39.808 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:39.808 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:39.808 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:39.808 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:39.808 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:39.808 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:39.808 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:39.808 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:39.808 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:39.808 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:39.809 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:39.809 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:39.809 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:39.809 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:39.809 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: Health check update: Degraded data redundancy: 85/630 objects degraded (13.492%), 27 pgs degraded (PG_DEGRADED) 2024-03-24T11:31:39.809 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:39.809 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:39.809 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:39.809 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:39.809 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:39.809 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:39.809 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:39.809 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:39.809 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:39.809 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:39 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:40.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:40.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:40.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:40.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:40.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:40.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:40.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:40.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:40.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:40.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:40.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:40.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T11:31:40.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: osd.7 [v2:172.21.15.139:6824/2848559150,v1:172.21.15.139:6825/2848559150] boot 2024-03-24T11:31:40.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: osdmap e172: 8 total, 8 up, 8 in 2024-03-24T11:31:40.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T11:31:40.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:40.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:40.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:40.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:40.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:40.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:40.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: pgmap v314: 161 pgs: 27 active+undersized+degraded, 42 active+undersized, 92 active+clean; 585 KiB data, 252 MiB used, 715 GiB / 715 GiB avail; 85/630 objects degraded (13.492%) 2024-03-24T11:31:40.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:40.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:40.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:40.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:40.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:40.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:40.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:40.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:40.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:40.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:40.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:40.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:40.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:40.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:40.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:40.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:40.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:40.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:40.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:40.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T11:31:40.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: osd.7 [v2:172.21.15.139:6824/2848559150,v1:172.21.15.139:6825/2848559150] boot 2024-03-24T11:31:40.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: osdmap e172: 8 total, 8 up, 8 in 2024-03-24T11:31:40.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T11:31:40.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:40.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:40.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:40.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:40.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:40.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:40.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: pgmap v314: 161 pgs: 27 active+undersized+degraded, 42 active+undersized, 92 active+clean; 585 KiB data, 252 MiB used, 715 GiB / 715 GiB avail; 85/630 objects degraded (13.492%) 2024-03-24T11:31:40.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:40.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:40.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:40.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:40.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:40.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:40.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:40.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:40.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:40.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:40.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:40.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:40.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:40.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:40.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:40.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:40.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:40.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:40.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:40.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:40.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:40.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:40.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:40.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:40.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:40.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:40.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:40.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:40.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:40.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:40.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:40.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:40.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:40.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:40.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:40.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:40.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:40.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:40.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:40.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:40.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:40.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:40.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:40.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:40.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:40.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:40.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:40.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:40.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:40.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:40.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:40.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:40.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:40.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:40.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:40.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:40.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:40.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:40.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:40.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:40.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:40.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:40.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:40.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:40.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:40.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:40.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:40.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:40.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:40.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:40.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:40.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:40.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:40.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:40.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:40.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:40.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:40.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:40.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:40.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:40.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:40.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:40.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:40.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:40.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:40.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:40.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:40.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:40.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:40.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:40.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:40.739 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:40.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:40.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:40.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:40.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:40.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:40.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:40.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:40.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:40.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:40.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:40.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:40.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-24T11:31:40.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: osd.7 [v2:172.21.15.139:6824/2848559150,v1:172.21.15.139:6825/2848559150] boot 2024-03-24T11:31:40.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: osdmap e172: 8 total, 8 up, 8 in 2024-03-24T11:31:40.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T11:31:40.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:40.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:40.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:40.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:40.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:40.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:40.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: pgmap v314: 161 pgs: 27 active+undersized+degraded, 42 active+undersized, 92 active+clean; 585 KiB data, 252 MiB used, 715 GiB / 715 GiB avail; 85/630 objects degraded (13.492%) 2024-03-24T11:31:40.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:40.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:40.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:40.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:40.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:40.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:40.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:40.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:40.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:40.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:40.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:40.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:40.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:40.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:40.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:40.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:40.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:40.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:40.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:40.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:40.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:40.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:40.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:40.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:40.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:40.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:40.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:40.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:40.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:40.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:40.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:40.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:40.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:40.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:40.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:40.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:40.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:40.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:40.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:40.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:40.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:40.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:40.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:40.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:40.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:40.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:40.863 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:40.864 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:40.864 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:40.864 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:40 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:41.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:31:40 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:31:40] "GET /metrics HTTP/1.1" 200 38615 "" "Prometheus/2.33.4" 2024-03-24T11:31:41.526 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:41.526 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:41.526 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:41.526 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:41.526 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:41.526 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:41.526 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:41.527 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:41.527 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:41.527 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:41.527 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:41.527 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:41.527 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:41.527 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:41.527 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:41.528 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:41.528 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: osdmap e173: 8 total, 8 up, 8 in 2024-03-24T11:31:41.528 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:41.528 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:41.528 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:41.528 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:41.528 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:41.528 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:41.528 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:41.528 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:41.528 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:41.528 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:41.528 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:41.529 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:41.529 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:41.529 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:41.529 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:41.529 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:41.529 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:41.529 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:41.529 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:41.529 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:41.529 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:41.529 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:41.529 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:41.529 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:41.530 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:41.530 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:41.530 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:41.530 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:41.530 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:41.530 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:41.530 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:41.530 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:41.530 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:41.530 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:41.530 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:41.530 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:41.530 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:41.531 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:31:41.531 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:41.531 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:41.531 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:41.531 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:41.531 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:41.531 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:41.531 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:41.531 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:41.531 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:41.531 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:41.531 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:41.531 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:41.531 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:41.532 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:41.532 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:41.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:41.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:41.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:41.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:41.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:41.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:41.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:41.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:41.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:41.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:41.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:41.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:41.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:41.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:41.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:41.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:41.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: osdmap e173: 8 total, 8 up, 8 in 2024-03-24T11:31:41.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:41.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:41.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:41.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:41.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:41.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:41.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:41.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:41.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:41.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:41.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:41.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:41.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:41.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:41.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:41.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:41.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:41.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:41.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:41.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:41.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:41.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:41.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:41.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:41.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:41.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:41.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:41.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:41.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:41.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: osdmap e173: 8 total, 8 up, 8 in 2024-03-24T11:31:41.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:41.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:41.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:41.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:41.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:41.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:41.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:41.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:41.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:41.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:41.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:41.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:41.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:41.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:41.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:41.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:41.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:41.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:41.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:41.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:41.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:41.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:41.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:41.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:41.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:41.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:41.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:41.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:41.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:41.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:41.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:41.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:41.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:41.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:41.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:41.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:41.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:41.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:31:41.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:41.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:41.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:41.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:41.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:41.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:41.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:41.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:41.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:41.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:41.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:41.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:41.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:41.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:41.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:41.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:41.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:41.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:41.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:41.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:41.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:41.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:41.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:41.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:41.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:41.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:41.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:41.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:41.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:41.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:31:41.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:41.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:41.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:41.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:41.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:41.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:41.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:41.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:41.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:41.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:41.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:41.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:41.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:41.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:41.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:41.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:41.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:41.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:41.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:41.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:41.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:41.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:41.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:41.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:41.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:42.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:42.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:42.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:42.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:42.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:42.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:42.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:42.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:42.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:42.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:42.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:42.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:42.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:42.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:42.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:42.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:42.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:42.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:42.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:42.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:42.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:42.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: pgmap v316: 161 pgs: 9 peering, 18 active+undersized+degraded, 31 active+undersized, 103 active+clean; 585 KiB data, 252 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 64/630 objects degraded (10.159%) 2024-03-24T11:31:42.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:42.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:42.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:42.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:42.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:42.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:42.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:42.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:42.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:42.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:42.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:42.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:42.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:42.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:42.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:42.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:42.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:42.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:42.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:42.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:42.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:42.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:42.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:42.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:42.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:42.616 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:42.616 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:42.616 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:42.616 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:42.616 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:42.616 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:42.617 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:42.617 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:42.617 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:42.617 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:42.617 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:42.617 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:42.618 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:42.618 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:42.618 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:42.618 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:42.618 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:42.618 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:42.618 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:42.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:42.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:42.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:42.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:42.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:42.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:42.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:42.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:42.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:42.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:42.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:42.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:42.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:42.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:42.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:42.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:42.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:42.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:42.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:42.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:42.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:42.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: pgmap v316: 161 pgs: 9 peering, 18 active+undersized+degraded, 31 active+undersized, 103 active+clean; 585 KiB data, 252 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 64/630 objects degraded (10.159%) 2024-03-24T11:31:42.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:42.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:42.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:42.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:42.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:42.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:42.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:42.729 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:42.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:42.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:42.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:42.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:42.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:42.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:42.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:42.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:42.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:42.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:42.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:42.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:42.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:42.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:42.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:42.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:42.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:42.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:42.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:42.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:42.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:42.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:42.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:42.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:42.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:42.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:42.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:42.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:42.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:42.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:42.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:42.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:42.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:42.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:42.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:42.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:42.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:42.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:42.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:42.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:42.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:42.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:42.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:42.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:42.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:42.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:42.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:42.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:42.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:42.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: pgmap v316: 161 pgs: 9 peering, 18 active+undersized+degraded, 31 active+undersized, 103 active+clean; 585 KiB data, 252 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 64/630 objects degraded (10.159%) 2024-03-24T11:31:42.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:42.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:42.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:42.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:42.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:42.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:42.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:42.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:42.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:42.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:42.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:42.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:42.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:42.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:42.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:42.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:42.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:42.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:42.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:42.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:42.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:42.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:42.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:42.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:42.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:42.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:42.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:42.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:42.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:42.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:42.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:42.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:42.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:42.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:42.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:42.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:42.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:42.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:42.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:42.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:42.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:42.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:42.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:42.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:42.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:42.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:42.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:42.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:42.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:42.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:42.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:42.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:43.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:43.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:43.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:43.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:43.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:43.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:43.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:43.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:43.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:43.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:43.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:31:43.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:43.609 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:43.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:43.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:43.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:43.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:43.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:43.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:43.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:43.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:43.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:43.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:43.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:43.610 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:43.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:43.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:43.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:43.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:43.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:43.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:43.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:43.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:43.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:43.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:43.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:43.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:43.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:43.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:43.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:43.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:43.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:43.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:43.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:43.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:43.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:43.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:43.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:43.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:43.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:43.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:43.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:43.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:43.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:43.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:43.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:43.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:43.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:43.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:43.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:43.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:43.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:43.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:43.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:43.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:43.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:43.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:43.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:43.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:43.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:43.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:43.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:43 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:43.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:43.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:43.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:43.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:43.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:43.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:43.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:43.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:43.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:43.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:43.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:31:43.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:43.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:43.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:43.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:43.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:43.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:43.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:43.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:43.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:43.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:43.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:43.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:43.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:43.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:43.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:43.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:43.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:43.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:43.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:43.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:43.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:43.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:43.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:43.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:43.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:43.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:43.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:43.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:43.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:43.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:31:43.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:43.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:43.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:43.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:43.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:43.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:43.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:43.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:43.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:43.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:43.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:43.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:43.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:43.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:43.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:43.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:43.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:43.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:43.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:43.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:43.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:43.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:43.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:43.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:43.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:43.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:43.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:43.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:43.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:43.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:43.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:43.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:43.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:43.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:43.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:43.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:43.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:43.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:43.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:43.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:43.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:43.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:43.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:43.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:43.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:43.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:43.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:43.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:43.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:43.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:43.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:43.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:43.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:43.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:43.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:43.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:43.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:43.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:43.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:43.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:43.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:43.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:43.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:43.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:43.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:43.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:43.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:43.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:43.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:43.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:43.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:43.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:43.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:43.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:43.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:43.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:43.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:43.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:43.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:43.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:43.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:43.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:43.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:43.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:43.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:43.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:43.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:43.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:43.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:43.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:43.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:43.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:43.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:43.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:43.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:43.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:43.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:43.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:43.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:43.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:43.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:43.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:43.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:43 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:44.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:44.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:44.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:44.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:44.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:44.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:44.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:44.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:44.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:44.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:44.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:44.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:44.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:44.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:44.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:44.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:44.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:44.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:44.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:44.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:44.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:44.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:44.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:44.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: pgmap v317: 161 pgs: 9 peering, 7 active+undersized+degraded, 15 active+undersized, 130 active+clean; 585 KiB data, 252 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 21/630 objects degraded (3.333%) 2024-03-24T11:31:44.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:44.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:44.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:44.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:44.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:44.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:44.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:44.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:44.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:44.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:44.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:44.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:44.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:44.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:44.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:44.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:44.730 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:44.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:44.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:44.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:44.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:44.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:44.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:44.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:44.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:44.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:44.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:44.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:44.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:44.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:44.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:44.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:44.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:44.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:44.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:44.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:44.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:44.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:44.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:44.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:44.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:44.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:44.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:44.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:44.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:44.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:44.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:44.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:44.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:44.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:44.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:44.733 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:44.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:44.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:44.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:44.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:44.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:44.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: pgmap v317: 161 pgs: 9 peering, 7 active+undersized+degraded, 15 active+undersized, 130 active+clean; 585 KiB data, 252 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 21/630 objects degraded (3.333%) 2024-03-24T11:31:44.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:44.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:44.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:44.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:44.734 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:44.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:44.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:44.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:44.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:44.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:44.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: Health check update: Degraded data redundancy: 64/630 objects degraded (10.159%), 18 pgs degraded (PG_DEGRADED) 2024-03-24T11:31:44.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:44.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:44.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:44.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:44.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:44.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:44.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:44.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:44.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:44.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:44.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:44.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:44.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:44.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:44.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:44.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:44.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:44.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:44.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:44.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:44.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:44.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:44.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:44.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:44.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:44.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:44.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:44.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:44.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:44.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:44.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:44.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:44.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:44.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:44.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:44.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:44.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:44.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:44.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:44.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: Health check update: Degraded data redundancy: 64/630 objects degraded (10.159%), 18 pgs degraded (PG_DEGRADED) 2024-03-24T11:31:44.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:44.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:44.739 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:44.739 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:44.739 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:44.800 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:44.800 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:44.800 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:44.800 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:44.800 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:44.801 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:44.801 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:44.801 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:44.801 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:44.801 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:44.801 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:44.801 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:44.801 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:44.801 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:44.801 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:44.801 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:44.801 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:44.801 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:44.801 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:44.801 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:44.801 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:44.802 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:44.802 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:44.802 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: pgmap v317: 161 pgs: 9 peering, 7 active+undersized+degraded, 15 active+undersized, 130 active+clean; 585 KiB data, 252 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 21/630 objects degraded (3.333%) 2024-03-24T11:31:44.802 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:44.802 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:44.802 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:44.802 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:44.802 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:44.802 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:44.802 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:44.802 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:44.802 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:44.802 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:44.802 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:44.803 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:44.803 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:44.803 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:44.803 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:44.803 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:44.803 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:44.803 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:44.803 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:44.803 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:44.803 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:44.803 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:44.803 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:44.803 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:44.803 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:44.803 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:44.804 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:44.804 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:44.804 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:44.804 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:44.804 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:44.804 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:44.804 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:44.804 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:44.804 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:44.804 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:44.804 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:44.804 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:44.804 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:44.804 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: Health check update: Degraded data redundancy: 64/630 objects degraded (10.159%), 18 pgs degraded (PG_DEGRADED) 2024-03-24T11:31:44.804 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:44.804 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:44.805 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:44.805 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:44.805 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:45.719 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:45.719 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:45.719 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:45.719 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:45.719 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:45.719 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:45.719 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:45.719 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:45.720 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:45.720 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:45.720 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:45.720 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:45.720 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:45.720 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:45.720 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:45.720 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:45.720 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:45.720 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:45.720 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:45.720 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:45.720 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:45.720 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:45.721 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:45.721 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:45.721 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:45.721 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:45.721 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:45.721 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:45.721 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:45.721 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:45.721 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:45.721 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:45.721 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:45.721 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:45.721 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:45.721 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:45.721 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:45.721 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:45.722 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:45.722 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:45.722 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:45.722 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:45.722 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:45.722 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:45.722 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:45.722 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:45.722 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:45.722 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:45.722 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:45.722 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:45.722 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:45.723 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:45.723 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:45.723 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:45.723 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:45.723 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:45.723 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:45.723 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:45.723 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:45.723 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:45.723 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:45.723 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:45.723 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:45.723 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:45.724 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:45.724 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:45.724 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:45.724 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:45.724 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:45.724 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:45.724 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:45 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:45.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:45.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:45.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:45.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:45.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:45.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:45.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:45.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:45.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:45.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:45.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:45.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:45.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:45.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:45.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:45.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:45.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:45.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:45.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:45.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:45.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:45.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:45.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:45.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:45.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:45.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:45.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:45.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:45.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:45.729 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:45.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:45.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:45.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:45.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:45.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:45.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:45.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:45.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:45.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:45.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:45.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:45.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:45.730 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:45.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:45.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:45.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:45.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:45.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:45.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:45.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:45.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:45.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:45.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:45.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:45.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:45.731 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:45.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:45.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:45.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:45.732 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:45.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:45.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:45.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:45.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:45.732 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:45.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:45.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:45.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:45.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:45.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:45.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:45.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:45.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:45.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:45.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:45.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:45.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:45.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:45.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:45.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:45.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:45.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:45.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:45.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:45.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:45.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:45.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:45.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:45.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:45.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:45.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:45.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:45.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:45.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:45.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:45.734 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:45.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:45.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:45.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:45.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:45.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:45.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:45.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:45.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:45.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:45.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:45.735 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:45.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:45.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:45.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:45.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:45.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:45.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:45.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:45.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:45.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:45.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:45.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:45.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:45.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:45.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:45.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:45.736 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:45.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:45.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:45.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:45.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:45.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:45.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:45.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:45.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:45.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:45.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:45.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:45.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:45.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:45.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:45.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:45.737 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:45.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:45.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:45.738 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:45 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:46.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:46.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:46.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:46.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:46.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:46.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:46.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:46.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:46.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:46.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:46.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:46.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:46.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:46.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:46.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:46.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:46.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:46.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:46.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:46.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:46.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:46.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:46.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:46.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: pgmap v318: 161 pgs: 161 active+clean; 585 KiB data, 252 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-03-24T11:31:46.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:46.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:46.479 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:46.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:46.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:46.480 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:46.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:46.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:46.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:46.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:46.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:46.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:46.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:46.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:46.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:46.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:46.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:46.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:46.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:46.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:46.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:46.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:46.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:46.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:46.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:46.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:46.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:46.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:46.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:46.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: pgmap v318: 161 pgs: 161 active+clean; 585 KiB data, 252 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-03-24T11:31:46.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:46.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:46.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:46.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:46.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:46.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:46.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:46.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:46.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:46.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:46.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:46.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:46.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:46.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:46.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:46.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:46.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:46.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:46.483 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:46.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:46.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:46.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:46.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:46.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:46.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:46.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:46.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:46.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:46.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:46.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:46.484 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:46.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:46.485 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:46.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:46.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:46.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:46.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:46.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:46.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:46.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:46.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:46.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:46.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:46.485 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:46.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:46.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:46.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:46.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:46.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:46.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:46.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:46.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:46.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:46.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:46.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:46.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:46.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:46.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:46.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:46.487 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:46 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:46.490 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:46.490 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:46.490 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:46.491 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:46.491 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:46.491 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:46.491 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:46.491 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:46.491 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:46.491 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:46.491 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:46.491 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:46.491 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:46.491 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:46.491 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:46.491 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:46.492 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:46.492 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:46.492 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:46.492 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:46.492 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:46.492 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:46.492 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:46.492 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: pgmap v318: 161 pgs: 161 active+clean; 585 KiB data, 252 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-03-24T11:31:46.492 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:46.492 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:46.492 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:46.492 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:46.493 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:46.493 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:46.493 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:46.493 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:46.493 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:46.493 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:46.493 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:46.493 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:46.493 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:46.493 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:46.493 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:46.493 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:46.493 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:46.493 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:46.493 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:46.493 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:46.494 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:46.494 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:46.494 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:46.494 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:46.494 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:46.494 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:46.494 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:46.494 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:46.494 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:46.494 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:46.494 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:46.494 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:46.494 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:46 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:46.978 INFO:teuthology.orchestra.run.smithi062.stdout:true 2024-03-24T11:31:47.593 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:47.593 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:47.593 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:47.593 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:47.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:47.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:47.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:47.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:47.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:47.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:47.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:47.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:47.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:47.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:47.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:47.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:47.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:47.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:47.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:47.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:47.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:47.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 21/630 objects degraded (3.333%), 7 pgs degraded) 2024-03-24T11:31:47.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:47.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:47.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:47.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:47.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:47.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:47.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:47.596 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:47.596 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:47.596 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:47.596 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:47.596 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:47.596 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:47.596 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:47.596 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:47.596 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:47.596 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:47.596 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:47.596 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:47.596 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:47.597 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:47.597 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:47.597 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:47.597 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:47.597 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:47.597 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:47.597 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:47.597 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:47.597 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 21/630 objects degraded (3.333%), 7 pgs degraded) 2024-03-24T11:31:47.597 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:47.597 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:47.598 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:47.598 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:47.598 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:47.598 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:47.598 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:47.598 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:47.598 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:47.598 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:47.598 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:47.598 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:47.599 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: from='client.64642 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:31:47.599 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:47.599 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:47.599 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:47.599 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:47.599 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:47.599 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:47.599 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:47.599 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:47.599 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:47.599 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:47.599 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:47.599 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:47.599 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:47.599 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:47.600 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:47.600 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:47.600 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:47.600 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:47.600 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:47.600 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:47.600 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:47.600 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:47.600 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:47.600 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:47.600 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:47.601 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:47.601 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: from='client.64642 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:31:47.601 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:47.601 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:47.601 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:47.601 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:47.601 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:47.601 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:47.601 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:47.601 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:47.601 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:47.601 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:47.601 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:47.601 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:47.602 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:47.602 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:47.602 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:47.602 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:47.602 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:47.602 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:47.602 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:47.602 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:47.602 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:47 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:47.727 INFO:teuthology.orchestra.run.smithi062.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T11:31:47.727 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi062 smithi062 running 17s ago 21m - - 2024-03-24T11:31:47.727 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi139 smithi139 running 18s ago 21m - - 2024-03-24T11:31:47.728 INFO:teuthology.orchestra.run.smithi062.stdout:alertmanager.a smithi062 *:9093,9094 running (11m) 17s ago 15m 25.3M - 0.23.0 ba2b418f427c 818298d58140 2024-03-24T11:31:47.728 INFO:teuthology.orchestra.run.smithi062.stdout:grafana.a smithi139 *:3000 running (10m) 18s ago 15m 53.9M - 8.3.5 dad864ee21e9 b48167e7171e 2024-03-24T11:31:47.728 INFO:teuthology.orchestra.run.smithi062.stdout:iscsi.foo.smithi062.dggsvk smithi062 running (9m) 17s ago 14m 66.2M - 3.5 e1d6a67b021e 0a98806ec839 2024-03-24T11:31:47.728 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.x smithi139 *:8443,9283,8765 running (8m) 18s ago 20m 453M - 19.0.0-1667-gdb0330b1 4f325f8440e9 d94987b11387 2024-03-24T11:31:47.728 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.y smithi062 *:8443,9283,8765 running (9m) 17s ago 22m 536M - 19.0.0-1667-gdb0330b1 4f325f8440e9 ca1a49028d65 2024-03-24T11:31:47.728 INFO:teuthology.orchestra.run.smithi062.stdout:mon.a smithi062 running (7m) 17s ago 22m 74.1M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 f7afe6e28c88 2024-03-24T11:31:47.728 INFO:teuthology.orchestra.run.smithi062.stdout:mon.b smithi139 running (6m) 18s ago 20m 54.4M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 ca357beea4dd 2024-03-24T11:31:47.728 INFO:teuthology.orchestra.run.smithi062.stdout:mon.c smithi062 running (7m) 17s ago 20m 63.5M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 459e7449ec39 2024-03-24T11:31:47.728 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.a smithi062 *:9100 running (11m) 17s ago 16m 15.3M - 1.3.1 1dbe0e931976 79d5a02a999f 2024-03-24T11:31:47.728 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.b smithi139 *:9100 running (10m) 18s ago 16m 15.7M - 1.3.1 1dbe0e931976 8f9f4608e465 2024-03-24T11:31:47.728 INFO:teuthology.orchestra.run.smithi062.stdout:osd.0 smithi062 running (5m) 17s ago 20m 61.4M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 dfe4920b49c4 2024-03-24T11:31:47.728 INFO:teuthology.orchestra.run.smithi062.stdout:osd.1 smithi062 running (4m) 17s ago 19m 54.4M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 c443b3fe97c1 2024-03-24T11:31:47.728 INFO:teuthology.orchestra.run.smithi062.stdout:osd.2 smithi062 running (3m) 17s ago 19m 47.9M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 0eba40dc96b3 2024-03-24T11:31:47.729 INFO:teuthology.orchestra.run.smithi062.stdout:osd.3 smithi062 running (2m) 17s ago 18m 57.0M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 7501d2e4c718 2024-03-24T11:31:47.729 INFO:teuthology.orchestra.run.smithi062.stdout:osd.4 smithi139 running (112s) 18s ago 18m 51.8M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 5d0f7f05ec6d 2024-03-24T11:31:47.729 INFO:teuthology.orchestra.run.smithi062.stdout:osd.5 smithi139 running (76s) 18s ago 18m 44.5M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 4ea95bfd2127 2024-03-24T11:31:47.729 INFO:teuthology.orchestra.run.smithi062.stdout:osd.6 smithi139 running (42s) 18s ago 17m 44.3M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 c67fd9426ac5 2024-03-24T11:31:47.729 INFO:teuthology.orchestra.run.smithi062.stdout:osd.7 smithi139 starting - - - 4096M 2024-03-24T11:31:47.729 INFO:teuthology.orchestra.run.smithi062.stdout:prometheus.a smithi139 *:9095 running (9m) 18s ago 16m 71.3M - 2.33.4 514e6a882f6e 2ed47f810d33 2024-03-24T11:31:47.729 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi062.sacpds smithi062 *:8000 running (15m) 17s ago 15m 75.5M - 17.2.0 e1d6a67b021e 89f4ac308e98 2024-03-24T11:31:47.729 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi139.yljesh smithi139 *:8000 running (14m) 18s ago 14m 74.4M - 17.2.0 e1d6a67b021e e189bdc12cae 2024-03-24T11:31:47.729 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi062.xvpxnk smithi062 *:80 running (14m) 17s ago 14m 75.3M - 17.2.0 e1d6a67b021e d1aa74adab44 2024-03-24T11:31:47.729 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi139.pbtrmp smithi139 *:80 running (14m) 18s ago 14m 78.2M - 17.2.0 e1d6a67b021e 7f8e336755d7 2024-03-24T11:31:47.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:47.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:47.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:47.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:47.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:47.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:47.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:47.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:47.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:47.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:47.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:47.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:47.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:47.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:47.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:47.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:47.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:47.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:47.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:47.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:47.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:47.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 21/630 objects degraded (3.333%), 7 pgs degraded) 2024-03-24T11:31:47.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:47.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:47.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:47.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:47.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:47.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:47.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:47.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:47.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:47.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:47.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:47.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:47.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: from='client.64642 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:31:47.860 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:47.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:47.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:47.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:47.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:47.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:47.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:47.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:47.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:47.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:47.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:47.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:47.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:47.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:47.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:47.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:47.861 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:47.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:47.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:47.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:47.862 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:47 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:48.239 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:31:48.239 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": { 2024-03-24T11:31:48.240 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 3 2024-03-24T11:31:48.240 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:31:48.240 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": { 2024-03-24T11:31:48.240 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-24T11:31:48.240 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:31:48.240 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": { 2024-03-24T11:31:48.240 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 8 2024-03-24T11:31:48.240 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:31:48.240 INFO:teuthology.orchestra.run.smithi062.stdout: "rgw": { 2024-03-24T11:31:48.240 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4 2024-03-24T11:31:48.240 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:31:48.240 INFO:teuthology.orchestra.run.smithi062.stdout: "overall": { 2024-03-24T11:31:48.240 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable)": 4, 2024-03-24T11:31:48.240 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 13 2024-03-24T11:31:48.241 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-24T11:31:48.241 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:31:48.599 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:31:48.599 INFO:teuthology.orchestra.run.smithi062.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69", 2024-03-24T11:31:48.599 INFO:teuthology.orchestra.run.smithi062.stdout: "in_progress": true, 2024-03-24T11:31:48.599 INFO:teuthology.orchestra.run.smithi062.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T11:31:48.599 INFO:teuthology.orchestra.run.smithi062.stdout: "services_complete": [ 2024-03-24T11:31:48.599 INFO:teuthology.orchestra.run.smithi062.stdout: "osd", 2024-03-24T11:31:48.599 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr", 2024-03-24T11:31:48.599 INFO:teuthology.orchestra.run.smithi062.stdout: "mon" 2024-03-24T11:31:48.600 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-03-24T11:31:48.600 INFO:teuthology.orchestra.run.smithi062.stdout: "progress": "13/23 daemons upgraded", 2024-03-24T11:31:48.600 INFO:teuthology.orchestra.run.smithi062.stdout: "message": "Currently upgrading osd daemons", 2024-03-24T11:31:48.600 INFO:teuthology.orchestra.run.smithi062.stdout: "is_paused": false 2024-03-24T11:31:48.600 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:31:48.600 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:48.600 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:48.600 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:48.600 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='client.44679 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:31:48.601 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:48.601 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:48.601 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:48.601 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:48.601 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:48.601 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:48.601 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:48.601 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:48.601 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:48.601 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:48.601 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:48.601 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:48.601 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='client.44685 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:31:48.601 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: pgmap v319: 161 pgs: 161 active+clean; 585 KiB data, 252 MiB used, 715 GiB / 715 GiB avail; 625 B/s rd, 0 op/s 2024-03-24T11:31:48.601 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:48.602 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:48.602 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:48.602 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:48.602 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:48.602 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:48.602 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:48.602 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:48.602 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:48.602 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:48.602 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:48.603 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:48.603 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:48.603 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:48.603 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='client.44679 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:31:48.603 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:48.603 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:48.603 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:48.603 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:48.603 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:48.603 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:48.603 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:48.604 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:48.604 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:48.604 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:48.604 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:48.604 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:48.604 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='client.44685 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:31:48.604 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: pgmap v319: 161 pgs: 161 active+clean; 585 KiB data, 252 MiB used, 715 GiB / 715 GiB avail; 625 B/s rd, 0 op/s 2024-03-24T11:31:48.604 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:48.604 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:48.604 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:48.604 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:48.604 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:48.604 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:48.604 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:48.604 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:48.605 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:48.605 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:48.605 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:48.605 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:48.605 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:48.605 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:48.605 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:48.605 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:48.605 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:48.605 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:48.605 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:48.605 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:48.605 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:48.605 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:48.606 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:48.606 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:48.606 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:48.606 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:48.606 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:48.606 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-03-24T11:31:48.606 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-03-24T11:31:48.606 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-03-24T11:31:48.606 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-03-24T11:31:48.606 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-03-24T11:31:48.606 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-03-24T11:31:48.606 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-03-24T11:31:48.606 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='client.? 172.21.15.62:0/302028762' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:48.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-03-24T11:31:48.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-03-24T11:31:48.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-03-24T11:31:48.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-03-24T11:31:48.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-03-24T11:31:48.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.6"}]: dispatch 2024-03-24T11:31:48.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.6"}]': finished 2024-03-24T11:31:48.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.7"}]: dispatch 2024-03-24T11:31:48.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.7"}]': finished 2024-03-24T11:31:48.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-03-24T11:31:48.608 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:48.608 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:48.608 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:48.608 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:48.608 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:48.608 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:48.608 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:48.608 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:48.608 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:48.608 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:48.608 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:48.608 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:48.608 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:48.609 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:48.609 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:48.609 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:48.609 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-03-24T11:31:48.609 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-03-24T11:31:48.609 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-03-24T11:31:48.609 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-03-24T11:31:48.609 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-03-24T11:31:48.609 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-03-24T11:31:48.609 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-03-24T11:31:48.609 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='client.? 172.21.15.62:0/302028762' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:48.609 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-03-24T11:31:48.609 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-03-24T11:31:48.609 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-03-24T11:31:48.610 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-03-24T11:31:48.610 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-03-24T11:31:48.610 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.6"}]: dispatch 2024-03-24T11:31:48.610 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.6"}]': finished 2024-03-24T11:31:48.610 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.7"}]: dispatch 2024-03-24T11:31:48.610 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.7"}]': finished 2024-03-24T11:31:48.610 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:48 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-03-24T11:31:48.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:48.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:48.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:48.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='client.44679 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:31:48.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:48.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:48.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:48.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:48.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:48.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:48.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:48.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:48.611 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:48.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:48.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:48.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:48.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='client.44685 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:31:48.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: pgmap v319: 161 pgs: 161 active+clean; 585 KiB data, 252 MiB used, 715 GiB / 715 GiB avail; 625 B/s rd, 0 op/s 2024-03-24T11:31:48.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:48.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:48.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:48.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:48.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:48.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:48.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:48.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:48.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:48.612 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:48.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:48.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:48.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:48.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:48.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:48.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:31:48.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:48.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:31:48.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:48.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:48.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:48.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:31:48.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:48.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:48.613 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:48.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:48.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:48.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-03-24T11:31:48.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-03-24T11:31:48.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-03-24T11:31:48.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-03-24T11:31:48.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-03-24T11:31:48.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-03-24T11:31:48.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-03-24T11:31:48.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='client.? 172.21.15.62:0/302028762' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:48.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-03-24T11:31:48.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-03-24T11:31:48.614 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-03-24T11:31:48.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-03-24T11:31:48.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-03-24T11:31:48.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.6"}]: dispatch 2024-03-24T11:31:48.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.6"}]': finished 2024-03-24T11:31:48.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.7"}]: dispatch 2024-03-24T11:31:48.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.7"}]': finished 2024-03-24T11:31:48.615 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:48 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-03-24T11:31:49.015 INFO:teuthology.orchestra.run.smithi062.stdout:HEALTH_WARN all OSDs are running squid or later but require_osd_release < squid 2024-03-24T11:31:49.015 INFO:teuthology.orchestra.run.smithi062.stdout:[WRN] OSD_UPGRADE_FINISHED: all OSDs are running squid or later but require_osd_release < squid 2024-03-24T11:31:49.016 INFO:teuthology.orchestra.run.smithi062.stdout: all OSDs are running squid or later but require_osd_release < squid 2024-03-24T11:31:49.720 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:49 smithi139 ceph-mon[91894]: Upgrade: Setting container_image for all osd 2024-03-24T11:31:49.720 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:49 smithi139 ceph-mon[91894]: Upgrade: Setting require_osd_release to 19 squid 2024-03-24T11:31:49.720 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:49 smithi139 ceph-mon[91894]: from='client.44694 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:31:49.721 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:49 smithi139 ceph-mon[91894]: from='client.? 172.21.15.62:0/4223893890' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:31:49.721 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:49.721 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:49.721 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:49.721 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:49 smithi139 ceph-mon[91894]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-03-24T11:31:49.721 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:49 smithi139 ceph-mon[91894]: Cluster is now healthy 2024-03-24T11:31:49.721 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-03-24T11:31:49.721 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:49 smithi139 ceph-mon[91894]: osdmap e174: 8 total, 8 up, 8 in 2024-03-24T11:31:49.721 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:49.721 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:49.898 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:49 smithi062 ceph-mon[90553]: Upgrade: Setting container_image for all osd 2024-03-24T11:31:49.898 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:49 smithi062 ceph-mon[90553]: Upgrade: Setting require_osd_release to 19 squid 2024-03-24T11:31:49.899 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:49 smithi062 ceph-mon[90553]: from='client.44694 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:31:49.899 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:49 smithi062 ceph-mon[90553]: from='client.? 172.21.15.62:0/4223893890' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:31:49.899 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:49.899 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:49.899 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:49.899 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:49 smithi062 ceph-mon[90553]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-03-24T11:31:49.899 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:49 smithi062 ceph-mon[90553]: Cluster is now healthy 2024-03-24T11:31:49.899 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-03-24T11:31:49.899 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:49 smithi062 ceph-mon[90553]: osdmap e174: 8 total, 8 up, 8 in 2024-03-24T11:31:49.899 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:49.899 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:49.899 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:49 smithi062 ceph-mon[98643]: Upgrade: Setting container_image for all osd 2024-03-24T11:31:49.899 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:49 smithi062 ceph-mon[98643]: Upgrade: Setting require_osd_release to 19 squid 2024-03-24T11:31:49.900 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:49 smithi062 ceph-mon[98643]: from='client.44694 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:31:49.900 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:49 smithi062 ceph-mon[98643]: from='client.? 172.21.15.62:0/4223893890' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:31:49.900 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:49.900 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:49.900 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:49.900 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:49 smithi062 ceph-mon[98643]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-03-24T11:31:49.900 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:49 smithi062 ceph-mon[98643]: Cluster is now healthy 2024-03-24T11:31:49.900 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-03-24T11:31:49.900 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:49 smithi062 ceph-mon[98643]: osdmap e174: 8 total, 8 up, 8 in 2024-03-24T11:31:49.900 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:31:49.900 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:50.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:50 smithi139 ceph-mon[91894]: Upgrade: Setting container_image for all mds 2024-03-24T11:31:50.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:50 smithi139 ceph-mon[91894]: pgmap v321: 161 pgs: 161 active+clean; 585 KiB data, 252 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-24T11:31:50.901 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:31:50 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:31:50] "GET /metrics HTTP/1.1" 200 38559 "" "Prometheus/2.33.4" 2024-03-24T11:31:50.902 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:50 smithi062 ceph-mon[90553]: Upgrade: Setting container_image for all mds 2024-03-24T11:31:50.902 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:50 smithi062 ceph-mon[90553]: pgmap v321: 161 pgs: 161 active+clean; 585 KiB data, 252 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-24T11:31:50.902 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:50 smithi062 ceph-mon[98643]: Upgrade: Setting container_image for all mds 2024-03-24T11:31:50.902 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:50 smithi062 ceph-mon[98643]: pgmap v321: 161 pgs: 161 active+clean; 585 KiB data, 252 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-24T11:31:51.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:51 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:51.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:51 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:51.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:51 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:51.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:51 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:51.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:51 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi062.sacpds", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-03-24T11:31:51.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:51 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:51.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:51 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:51.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:51 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:51.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:51 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:51.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:51 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:51.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:51 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:51.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:51 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi062.sacpds", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-03-24T11:31:51.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:51 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:51.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:51 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:51.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:51 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:51.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:51 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:51.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:51 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:51.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:51 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:51.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:51 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi062.sacpds", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-03-24T11:31:51.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:51 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:51.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:51 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:52.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:52 smithi062 ceph-mon[90553]: Upgrade: Updating rgw.foo.smithi062.sacpds (1/4) 2024-03-24T11:31:52.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:52 smithi062 ceph-mon[90553]: Deploying daemon rgw.foo.smithi062.sacpds on smithi062 2024-03-24T11:31:52.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:52 smithi062 ceph-mon[90553]: pgmap v322: 161 pgs: 161 active+clean; 585 KiB data, 252 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T11:31:52.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:52 smithi062 ceph-mon[98643]: Upgrade: Updating rgw.foo.smithi062.sacpds (1/4) 2024-03-24T11:31:52.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:52 smithi062 ceph-mon[98643]: Deploying daemon rgw.foo.smithi062.sacpds on smithi062 2024-03-24T11:31:52.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:52 smithi062 ceph-mon[98643]: pgmap v322: 161 pgs: 161 active+clean; 585 KiB data, 252 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T11:31:52.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:52 smithi139 ceph-mon[91894]: Upgrade: Updating rgw.foo.smithi062.sacpds (1/4) 2024-03-24T11:31:52.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:52 smithi139 ceph-mon[91894]: Deploying daemon rgw.foo.smithi062.sacpds on smithi062 2024-03-24T11:31:52.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:52 smithi139 ceph-mon[91894]: pgmap v322: 161 pgs: 161 active+clean; 585 KiB data, 252 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T11:31:53.920 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:53 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:31:53.920 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:53 smithi062 ceph-mon[98643]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:31:54.017 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:53 smithi139 ceph-mon[91894]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:31:54.968 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:54 smithi062 ceph-mon[90553]: pgmap v323: 161 pgs: 161 active+clean; 585 KiB data, 252 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T11:31:54.969 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:54 smithi062 ceph-mon[98643]: pgmap v323: 161 pgs: 161 active+clean; 585 KiB data, 252 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T11:31:55.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:54 smithi139 ceph-mon[91894]: pgmap v323: 161 pgs: 161 active+clean; 585 KiB data, 252 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T11:31:56.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:56.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:56.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:56 smithi139 ceph-mon[91894]: pgmap v324: 161 pgs: 161 active+clean; 585 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-03-24T11:31:56.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:56.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:31:56.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:56.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:56.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:56 smithi062 ceph-mon[98643]: pgmap v324: 161 pgs: 161 active+clean; 585 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-03-24T11:31:56.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:56.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:31:56.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:56.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:56.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:56 smithi062 ceph-mon[90553]: pgmap v324: 161 pgs: 161 active+clean; 585 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-03-24T11:31:56.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:56.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:31:58.446 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:58 smithi062 ceph-mon[98643]: Upgrade: Updating rgw.smpl.smithi062.xvpxnk (2/4) 2024-03-24T11:31:58.447 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:58.447 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi062.xvpxnk", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-03-24T11:31:58.447 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:58.447 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:58 smithi062 ceph-mon[98643]: Deploying daemon rgw.smpl.smithi062.xvpxnk on smithi062 2024-03-24T11:31:58.447 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:31:58 smithi062 ceph-mon[98643]: pgmap v325: 161 pgs: 161 active+clean; 585 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 11 KiB/s rd, 0 B/s wr, 16 op/s 2024-03-24T11:31:58.447 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:58 smithi062 ceph-mon[90553]: Upgrade: Updating rgw.smpl.smithi062.xvpxnk (2/4) 2024-03-24T11:31:58.447 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:58.447 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi062.xvpxnk", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-03-24T11:31:58.447 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:58.447 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:58 smithi062 ceph-mon[90553]: Deploying daemon rgw.smpl.smithi062.xvpxnk on smithi062 2024-03-24T11:31:58.447 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:31:58 smithi062 ceph-mon[90553]: pgmap v325: 161 pgs: 161 active+clean; 585 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 11 KiB/s rd, 0 B/s wr, 16 op/s 2024-03-24T11:31:58.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:58 smithi139 ceph-mon[91894]: Upgrade: Updating rgw.smpl.smithi062.xvpxnk (2/4) 2024-03-24T11:31:58.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:31:58.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi062.xvpxnk", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-03-24T11:31:58.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:31:58.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:58 smithi139 ceph-mon[91894]: Deploying daemon rgw.smpl.smithi062.xvpxnk on smithi062 2024-03-24T11:31:58.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:31:58 smithi139 ceph-mon[91894]: pgmap v325: 161 pgs: 161 active+clean; 585 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 11 KiB/s rd, 0 B/s wr, 16 op/s 2024-03-24T11:32:00.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:00 smithi062 ceph-mon[90553]: pgmap v326: 161 pgs: 161 active+clean; 585 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 51 KiB/s rd, 97 B/s wr, 78 op/s 2024-03-24T11:32:00.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:00 smithi062 ceph-mon[98643]: pgmap v326: 161 pgs: 161 active+clean; 585 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 51 KiB/s rd, 97 B/s wr, 78 op/s 2024-03-24T11:32:00.977 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:32:00 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:32:00] "GET /metrics HTTP/1.1" 200 38696 "" "Prometheus/2.33.4" 2024-03-24T11:32:01.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:00 smithi139 ceph-mon[91894]: pgmap v326: 161 pgs: 161 active+clean; 585 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 51 KiB/s rd, 97 B/s wr, 78 op/s 2024-03-24T11:32:02.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:02.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:02 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:02.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:02 smithi062 ceph-mon[90553]: pgmap v327: 161 pgs: 161 active+clean; 585 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 55 KiB/s rd, 85 B/s wr, 84 op/s 2024-03-24T11:32:02.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:02.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:02 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:02.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:02 smithi062 ceph-mon[98643]: pgmap v327: 161 pgs: 161 active+clean; 585 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 55 KiB/s rd, 85 B/s wr, 84 op/s 2024-03-24T11:32:02.733 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:02.733 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:02 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:02.733 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:02 smithi139 ceph-mon[91894]: pgmap v327: 161 pgs: 161 active+clean; 585 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 55 KiB/s rd, 85 B/s wr, 84 op/s 2024-03-24T11:32:03.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:03 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:32:03.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:03.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi139.yljesh", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-03-24T11:32:03.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:03 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:03.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:03 smithi062 ceph-mon[98643]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:32:03.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:03.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi139.yljesh", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-03-24T11:32:03.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:03 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:03.751 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:03 smithi139 ceph-mon[91894]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:32:03.751 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:03.752 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi139.yljesh", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-03-24T11:32:03.752 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:03 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:04.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:04 smithi062 ceph-mon[90553]: Upgrade: Updating rgw.foo.smithi139.yljesh (3/4) 2024-03-24T11:32:04.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:04 smithi062 ceph-mon[90553]: Deploying daemon rgw.foo.smithi139.yljesh on smithi139 2024-03-24T11:32:04.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:04 smithi062 ceph-mon[90553]: pgmap v328: 161 pgs: 161 active+clean; 585 KiB data, 261 MiB used, 715 GiB / 715 GiB avail; 74 KiB/s rd, 170 B/s wr, 113 op/s 2024-03-24T11:32:04.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:04 smithi062 ceph-mon[98643]: Upgrade: Updating rgw.foo.smithi139.yljesh (3/4) 2024-03-24T11:32:04.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:04 smithi062 ceph-mon[98643]: Deploying daemon rgw.foo.smithi139.yljesh on smithi139 2024-03-24T11:32:04.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:04 smithi062 ceph-mon[98643]: pgmap v328: 161 pgs: 161 active+clean; 585 KiB data, 261 MiB used, 715 GiB / 715 GiB avail; 74 KiB/s rd, 170 B/s wr, 113 op/s 2024-03-24T11:32:04.754 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:04 smithi139 ceph-mon[91894]: Upgrade: Updating rgw.foo.smithi139.yljesh (3/4) 2024-03-24T11:32:04.754 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:04 smithi139 ceph-mon[91894]: Deploying daemon rgw.foo.smithi139.yljesh on smithi139 2024-03-24T11:32:04.754 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:04 smithi139 ceph-mon[91894]: pgmap v328: 161 pgs: 161 active+clean; 585 KiB data, 261 MiB used, 715 GiB / 715 GiB avail; 74 KiB/s rd, 170 B/s wr, 113 op/s 2024-03-24T11:32:06.947 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:06 smithi062 ceph-mon[90553]: pgmap v329: 161 pgs: 161 active+clean; 585 KiB data, 261 MiB used, 715 GiB / 715 GiB avail; 101 KiB/s rd, 170 B/s wr, 155 op/s 2024-03-24T11:32:06.947 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:06 smithi062 ceph-mon[98643]: pgmap v329: 161 pgs: 161 active+clean; 585 KiB data, 261 MiB used, 715 GiB / 715 GiB avail; 101 KiB/s rd, 170 B/s wr, 155 op/s 2024-03-24T11:32:07.027 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:06 smithi139 ceph-mon[91894]: pgmap v329: 161 pgs: 161 active+clean; 585 KiB data, 261 MiB used, 715 GiB / 715 GiB avail; 101 KiB/s rd, 170 B/s wr, 155 op/s 2024-03-24T11:32:09.029 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:08 smithi062 ceph-mon[90553]: pgmap v330: 161 pgs: 161 active+clean; 585 KiB data, 261 MiB used, 715 GiB / 715 GiB avail; 109 KiB/s rd, 170 B/s wr, 168 op/s 2024-03-24T11:32:09.030 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:09.030 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:08 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:09.030 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:08 smithi062 ceph-mon[98643]: pgmap v330: 161 pgs: 161 active+clean; 585 KiB data, 261 MiB used, 715 GiB / 715 GiB avail; 109 KiB/s rd, 170 B/s wr, 168 op/s 2024-03-24T11:32:09.030 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:09.030 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:08 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:09.282 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:08 smithi139 ceph-mon[91894]: pgmap v330: 161 pgs: 161 active+clean; 585 KiB data, 261 MiB used, 715 GiB / 715 GiB avail; 109 KiB/s rd, 170 B/s wr, 168 op/s 2024-03-24T11:32:09.282 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:09.282 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:08 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:10.579 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:10.579 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:10.580 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:10.580 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:10 smithi139 ceph-mon[91894]: pgmap v331: 161 pgs: 161 active+clean; 585 KiB data, 261 MiB used, 715 GiB / 715 GiB avail; 135 KiB/s rd, 255 B/s wr, 207 op/s 2024-03-24T11:32:10.618 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:10.618 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:10.618 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:10.618 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:10 smithi062 ceph-mon[90553]: pgmap v331: 161 pgs: 161 active+clean; 585 KiB data, 261 MiB used, 715 GiB / 715 GiB avail; 135 KiB/s rd, 255 B/s wr, 207 op/s 2024-03-24T11:32:10.618 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:10.618 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:10.618 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:10.618 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:10 smithi062 ceph-mon[98643]: pgmap v331: 161 pgs: 161 active+clean; 585 KiB data, 261 MiB used, 715 GiB / 715 GiB avail; 135 KiB/s rd, 255 B/s wr, 207 op/s 2024-03-24T11:32:10.977 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:32:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:32:10] "GET /metrics HTTP/1.1" 200 38696 "" "Prometheus/2.33.4" 2024-03-24T11:32:11.640 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:11 smithi062 ceph-mon[90553]: Upgrade: Updating rgw.smpl.smithi139.pbtrmp (4/4) 2024-03-24T11:32:11.641 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:11.641 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi139.pbtrmp", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-03-24T11:32:11.641 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:11.641 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:11 smithi062 ceph-mon[90553]: Deploying daemon rgw.smpl.smithi139.pbtrmp on smithi139 2024-03-24T11:32:11.641 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:11.641 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:11.641 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:11.641 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:32:11.641 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:11 smithi062 ceph-mon[98643]: Upgrade: Updating rgw.smpl.smithi139.pbtrmp (4/4) 2024-03-24T11:32:11.641 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:11.641 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi139.pbtrmp", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-03-24T11:32:11.642 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:11.642 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:11 smithi062 ceph-mon[98643]: Deploying daemon rgw.smpl.smithi139.pbtrmp on smithi139 2024-03-24T11:32:11.642 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:11.642 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:11.642 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:11.642 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:32:11.845 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:11 smithi139 ceph-mon[91894]: Upgrade: Updating rgw.smpl.smithi139.pbtrmp (4/4) 2024-03-24T11:32:11.845 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:11.845 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi139.pbtrmp", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-03-24T11:32:11.845 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:11.845 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:11 smithi139 ceph-mon[91894]: Deploying daemon rgw.smpl.smithi139.pbtrmp on smithi139 2024-03-24T11:32:11.845 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:11.845 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:11.846 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:11.846 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:32:12.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:12 smithi062 ceph-mon[90553]: pgmap v332: 161 pgs: 161 active+clean; 585 KiB data, 265 MiB used, 715 GiB / 715 GiB avail; 117 KiB/s rd, 170 B/s wr, 181 op/s 2024-03-24T11:32:12.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:12 smithi062 ceph-mon[98643]: pgmap v332: 161 pgs: 161 active+clean; 585 KiB data, 265 MiB used, 715 GiB / 715 GiB avail; 117 KiB/s rd, 170 B/s wr, 181 op/s 2024-03-24T11:32:12.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:12 smithi139 ceph-mon[91894]: pgmap v332: 161 pgs: 161 active+clean; 585 KiB data, 265 MiB used, 715 GiB / 715 GiB avail; 117 KiB/s rd, 170 B/s wr, 181 op/s 2024-03-24T11:32:14.188 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:13 smithi139 ceph-mon[91894]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:32:14.188 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:14.188 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:14.189 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:13 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:14.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:13 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:32:14.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:14.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:14.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:13 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:14.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:13 smithi062 ceph-mon[98643]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:32:14.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:14.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:14.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:13 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:15.218 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:14 smithi139 ceph-mon[91894]: pgmap v333: 161 pgs: 161 active+clean; 585 KiB data, 265 MiB used, 715 GiB / 715 GiB avail; 109 KiB/s rd, 170 B/s wr, 168 op/s 2024-03-24T11:32:15.219 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:14 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:15.219 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:14 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:15.219 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:14 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:15.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:14 smithi062 ceph-mon[90553]: pgmap v333: 161 pgs: 161 active+clean; 585 KiB data, 265 MiB used, 715 GiB / 715 GiB avail; 109 KiB/s rd, 170 B/s wr, 168 op/s 2024-03-24T11:32:15.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:14 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:15.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:14 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:15.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:14 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:15.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:14 smithi062 ceph-mon[98643]: pgmap v333: 161 pgs: 161 active+clean; 585 KiB data, 265 MiB used, 715 GiB / 715 GiB avail; 109 KiB/s rd, 170 B/s wr, 168 op/s 2024-03-24T11:32:15.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:14 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:15.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:14 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:15.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:14 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:17.009 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:16 smithi139 ceph-mon[91894]: pgmap v334: 161 pgs: 161 active+clean; 585 KiB data, 265 MiB used, 715 GiB / 715 GiB avail; 91 KiB/s rd, 85 B/s wr, 139 op/s 2024-03-24T11:32:17.009 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:17.009 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:17.009 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:17.010 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:16 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:17.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:16 smithi062 ceph-mon[90553]: pgmap v334: 161 pgs: 161 active+clean; 585 KiB data, 265 MiB used, 715 GiB / 715 GiB avail; 91 KiB/s rd, 85 B/s wr, 139 op/s 2024-03-24T11:32:17.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:17.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:17.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:17.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:16 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:17.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:16 smithi062 ceph-mon[98643]: pgmap v334: 161 pgs: 161 active+clean; 585 KiB data, 265 MiB used, 715 GiB / 715 GiB avail; 91 KiB/s rd, 85 B/s wr, 139 op/s 2024-03-24T11:32:17.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:17.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:17.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:17.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:16 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:18.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:18.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:17 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:18.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:18.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:18.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:18.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:18.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:17 smithi139 ceph-mon[91894]: Checking dashboard <-> RGW credentials 2024-03-24T11:32:18.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:18.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:18.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:18.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:18.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:18.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:18.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:18.114 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:18.115 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:18.115 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:18.115 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:18.115 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:18.115 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:18.115 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:18.115 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:18.115 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:18.115 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:18.115 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:18.115 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:18.115 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:18.115 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:18.115 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:18.115 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:18.116 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:18.116 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:18.116 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:18.116 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:18.116 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:18.116 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:18.116 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:18.116 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:18.116 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:18.116 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:18.116 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:18.116 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:18.116 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:18.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:18.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:18.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:18.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:18.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:18.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:18.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[90553]: Checking dashboard <-> RGW credentials 2024-03-24T11:32:18.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:18.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:18.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:18.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:18.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:18.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:18.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:18.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:18.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:18.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:18.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:18.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:18.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:18.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:18.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:18.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:18.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:18.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:18.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:18.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:18.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:18.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:18.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:18.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:18.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:18.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:18.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:18.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:18.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[98643]: Checking dashboard <-> RGW credentials 2024-03-24T11:32:18.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:18.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:18.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:18.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:18.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:18.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:18.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:18.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:18.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:18.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:18.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:18.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:18.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:18.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:18.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:18.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:18.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:18.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:18.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:18.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:18.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:18.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:18.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:18.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:18.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:18.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:18.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:18.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:18.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:18.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:18.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:18.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:18.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:18.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:18.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:18.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:18.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:18.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:18.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:18.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:18.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:18.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:18.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:18.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:18.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:18.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:18.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:18.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:18.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:18.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:19.017 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:19.017 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:19.017 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:19.018 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:19.018 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:19.018 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:19.018 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: pgmap v335: 161 pgs: 161 active+clean; 585 KiB data, 265 MiB used, 715 GiB / 715 GiB avail; 64 KiB/s rd, 85 B/s wr, 97 op/s 2024-03-24T11:32:19.018 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:19.018 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:19.018 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:19.018 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:19.018 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:19.018 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:19.018 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:19.018 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:19.018 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:19.018 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:19.019 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:19.019 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:19.019 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:19.019 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:19.019 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:19.019 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:19.019 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:19.019 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:19.019 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:19.019 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:19.019 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:19.019 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:19.019 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:19.019 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:19.020 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:19.020 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:19.020 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:19.020 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:19.020 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:19.020 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:19.020 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:19.020 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:19.020 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:19.020 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:19.020 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:19.020 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:19.020 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:19.020 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:19.020 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:19.021 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:19.021 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:19.021 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:19.021 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:19.021 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:19.021 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:19.021 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:19.021 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:19.021 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:19.021 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:19.021 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:19.021 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:19.021 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:19.021 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:19.021 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:19.022 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:19.022 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:19.022 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:19.022 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:19.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:19.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:19.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:19.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:19.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:19.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:19.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: pgmap v335: 161 pgs: 161 active+clean; 585 KiB data, 265 MiB used, 715 GiB / 715 GiB avail; 64 KiB/s rd, 85 B/s wr, 97 op/s 2024-03-24T11:32:19.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:19.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:19.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:19.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:19.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:19.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:19.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:19.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:19.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:19.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:19.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:19.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:19.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:19.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:19.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:19.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:19.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:19.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:19.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:19.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:19.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:19.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:19.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:19.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:19.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:19.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:19.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:19.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:19.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:19.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:19.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:19.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:19.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:19.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:19.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:19.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:19.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:19.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:19.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:19.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:19.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:19.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:19.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:19.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:19.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:19.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:19.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:19.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:19.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:19.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:19.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:19.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:19.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:19.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:19.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:19.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:19.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:19.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: pgmap v335: 161 pgs: 161 active+clean; 585 KiB data, 265 MiB used, 715 GiB / 715 GiB avail; 64 KiB/s rd, 85 B/s wr, 97 op/s 2024-03-24T11:32:19.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:19.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:19.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:19.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:19.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:19.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:19.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:19.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:19.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:19.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:19.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:19.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:19.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:19.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:19.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:19.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:19.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:19.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:19.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:19.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:19.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:19.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:19.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:19.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:19.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:19.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:19.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:19.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:19.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:19.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:19.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:19.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:19.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:19.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:19.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:19.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:19.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:19.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:19.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:19.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:19.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:19.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:19.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:19.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:19.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:19.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:19.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:19.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:19.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:19.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:19.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:19.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:19.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:19.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:19.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:19.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:19.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:19.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:19.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:19.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:19.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:19.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:19.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:19.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:19.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:19.395 INFO:teuthology.orchestra.run.smithi062.stdout:true 2024-03-24T11:32:20.093 INFO:teuthology.orchestra.run.smithi062.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T11:32:20.093 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi062 smithi062 running 7s ago 21m - - 2024-03-24T11:32:20.094 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi139 smithi139 running 5s ago 21m - - 2024-03-24T11:32:20.094 INFO:teuthology.orchestra.run.smithi062.stdout:alertmanager.a smithi062 *:9093,9094 running (11m) 7s ago 16m 25.3M - 0.23.0 ba2b418f427c 818298d58140 2024-03-24T11:32:20.094 INFO:teuthology.orchestra.run.smithi062.stdout:grafana.a smithi139 *:3000 running (10m) 5s ago 16m 54.9M - 8.3.5 dad864ee21e9 b48167e7171e 2024-03-24T11:32:20.094 INFO:teuthology.orchestra.run.smithi062.stdout:iscsi.foo.smithi062.dggsvk smithi062 running (9m) 7s ago 15m 66.2M - 3.5 e1d6a67b021e 0a98806ec839 2024-03-24T11:32:20.094 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.x smithi139 *:8443,9283,8765 running (9m) 5s ago 20m 453M - 19.0.0-1667-gdb0330b1 4f325f8440e9 d94987b11387 2024-03-24T11:32:20.094 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.y smithi062 *:8443,9283,8765 running (10m) 7s ago 22m 538M - 19.0.0-1667-gdb0330b1 4f325f8440e9 ca1a49028d65 2024-03-24T11:32:20.094 INFO:teuthology.orchestra.run.smithi062.stdout:mon.a smithi062 running (8m) 7s ago 22m 75.1M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 f7afe6e28c88 2024-03-24T11:32:20.094 INFO:teuthology.orchestra.run.smithi062.stdout:mon.b smithi139 running (7m) 5s ago 21m 54.3M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 ca357beea4dd 2024-03-24T11:32:20.094 INFO:teuthology.orchestra.run.smithi062.stdout:mon.c smithi062 running (7m) 7s ago 21m 63.0M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 459e7449ec39 2024-03-24T11:32:20.094 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.a smithi062 *:9100 running (11m) 7s ago 17m 15.5M - 1.3.1 1dbe0e931976 79d5a02a999f 2024-03-24T11:32:20.094 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.b smithi139 *:9100 running (11m) 5s ago 17m 15.7M - 1.3.1 1dbe0e931976 8f9f4608e465 2024-03-24T11:32:20.094 INFO:teuthology.orchestra.run.smithi062.stdout:osd.0 smithi062 running (6m) 7s ago 20m 64.8M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 dfe4920b49c4 2024-03-24T11:32:20.094 INFO:teuthology.orchestra.run.smithi062.stdout:osd.1 smithi062 running (5m) 7s ago 20m 57.3M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 c443b3fe97c1 2024-03-24T11:32:20.094 INFO:teuthology.orchestra.run.smithi062.stdout:osd.2 smithi062 running (4m) 7s ago 19m 52.0M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 0eba40dc96b3 2024-03-24T11:32:20.095 INFO:teuthology.orchestra.run.smithi062.stdout:osd.3 smithi062 running (3m) 7s ago 19m 62.1M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 7501d2e4c718 2024-03-24T11:32:20.095 INFO:teuthology.orchestra.run.smithi062.stdout:osd.4 smithi139 running (2m) 5s ago 19m 55.7M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 5d0f7f05ec6d 2024-03-24T11:32:20.095 INFO:teuthology.orchestra.run.smithi062.stdout:osd.5 smithi139 running (109s) 5s ago 18m 47.5M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 4ea95bfd2127 2024-03-24T11:32:20.095 INFO:teuthology.orchestra.run.smithi062.stdout:osd.6 smithi139 running (75s) 5s ago 18m 47.4M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 c67fd9426ac5 2024-03-24T11:32:20.095 INFO:teuthology.orchestra.run.smithi062.stdout:osd.7 smithi139 running (46s) 5s ago 17m 46.9M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 d4fbd9bccd09 2024-03-24T11:32:20.095 INFO:teuthology.orchestra.run.smithi062.stdout:prometheus.a smithi139 *:9095 running (10m) 5s ago 17m 71.3M - 2.33.4 514e6a882f6e 2ed47f810d33 2024-03-24T11:32:20.095 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi062.sacpds smithi062 *:8000 running (25s) 7s ago 15m 75.1M - 19.0.0-1667-gdb0330b1 4f325f8440e9 bdccb278a0c3 2024-03-24T11:32:20.095 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi139.yljesh smithi139 *:8000 running (12s) 5s ago 15m - - 19.0.0-1667-gdb0330b1 4f325f8440e9 9c325556e724 2024-03-24T11:32:20.095 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi062.xvpxnk smithi062 *:80 running (19s) 7s ago 15m 76.3M - 19.0.0-1667-gdb0330b1 4f325f8440e9 88bcf6d3dca0 2024-03-24T11:32:20.095 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi139.pbtrmp smithi139 *:80 starting - - - - 2024-03-24T11:32:20.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:19 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:20.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:19 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:20.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:19 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:20.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:19 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:20.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:19 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:20.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:19 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:20.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:19 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:20.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:19 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:20.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:19 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:20.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:19 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:20.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:19 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:20.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:19 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:20.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:19 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:20.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:19 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:20.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:19 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:20.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:19 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:20.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:19 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:20.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:19 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:20.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:19 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:20.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:19 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:20.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:19 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:20.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:19 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:20.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:19 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:20.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:19 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:20.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:19 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:20.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:19 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:20.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:19 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:20.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:19 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:20.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:19 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:20.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:19 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:20.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:19 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:20.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:19 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:20.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:19 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:20.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:19 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:20.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:19 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:20.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:19 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:20.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:19 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:20.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:19 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:20.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:19 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:20.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:19 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:20.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:19 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:20.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:19 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:20.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:19 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:20.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:19 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:20.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:19 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:20.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:19 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:20.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:19 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:20.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:19 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:20.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:19 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:20.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:19 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:20.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:19 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:20.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:20.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:20.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:20.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:20.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:20.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:20.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:20.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:20.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:20.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:20.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:20.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:20.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:20.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:20.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:20.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:20.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:20.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:20.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:20.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:20.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:20.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:20.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:20.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:20.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:20.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:20.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:20.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:20.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:20.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:20.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:20.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:20.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:20.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:20.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:20.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:20.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:20.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:20.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:20.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:20.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:20.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:20.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:20.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:20.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:20.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:20.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:20.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:20.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:20.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:20.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:20.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:20.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:20.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:20.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:20.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:20.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:20.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:20.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:20.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:20.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:20.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:20.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:20.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:20.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:20.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:20.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:20.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:20.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:20.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:20.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:20.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:20.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:20.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:20.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:20.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:20.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:20.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:20.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:20.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:20.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:20.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:20.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:20.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:20.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:20.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:20.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:20.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:20.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:20.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:20.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:20.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:20.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:20.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:20.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:20.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:20.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:20.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:20.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:20.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:20.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:20.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:19 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:20.515 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:32:20.515 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": { 2024-03-24T11:32:20.515 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 3 2024-03-24T11:32:20.515 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:32:20.515 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": { 2024-03-24T11:32:20.515 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-24T11:32:20.515 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:32:20.515 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": { 2024-03-24T11:32:20.515 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 8 2024-03-24T11:32:20.515 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:32:20.515 INFO:teuthology.orchestra.run.smithi062.stdout: "rgw": { 2024-03-24T11:32:20.516 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 4 2024-03-24T11:32:20.516 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:32:20.516 INFO:teuthology.orchestra.run.smithi062.stdout: "overall": { 2024-03-24T11:32:20.516 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 17 2024-03-24T11:32:20.516 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-24T11:32:20.516 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:32:20.870 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:32:20.870 INFO:teuthology.orchestra.run.smithi062.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69", 2024-03-24T11:32:20.870 INFO:teuthology.orchestra.run.smithi062.stdout: "in_progress": true, 2024-03-24T11:32:20.870 INFO:teuthology.orchestra.run.smithi062.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T11:32:20.870 INFO:teuthology.orchestra.run.smithi062.stdout: "services_complete": [ 2024-03-24T11:32:20.871 INFO:teuthology.orchestra.run.smithi062.stdout: "osd", 2024-03-24T11:32:20.871 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr", 2024-03-24T11:32:20.871 INFO:teuthology.orchestra.run.smithi062.stdout: "mon" 2024-03-24T11:32:20.871 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-03-24T11:32:20.871 INFO:teuthology.orchestra.run.smithi062.stdout: "progress": "16/23 daemons upgraded", 2024-03-24T11:32:20.871 INFO:teuthology.orchestra.run.smithi062.stdout: "message": "Currently upgrading rgw daemons", 2024-03-24T11:32:20.871 INFO:teuthology.orchestra.run.smithi062.stdout: "is_paused": false 2024-03-24T11:32:20.871 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:32:21.070 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:20 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:21.070 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:20 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:21.070 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:20 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:21.070 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:20 smithi139 ceph-mon[91894]: from='client.64795 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:32:21.070 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:20 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:21.070 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:20 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:21.070 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:20 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:21.070 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:20 smithi139 ceph-mon[91894]: from='client.64801 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:32:21.070 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:20 smithi139 ceph-mon[91894]: pgmap v336: 161 pgs: 161 active+clean; 585 KiB data, 273 MiB used, 715 GiB / 715 GiB avail; 92 KiB/s rd, 170 B/s wr, 140 op/s 2024-03-24T11:32:21.070 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:20 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:21.070 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:21.071 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:21.071 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:21.071 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:21.071 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:21.071 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:20 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:21.071 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:21.071 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:21.071 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:21.071 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:21.071 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:21.071 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:21.071 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:21.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:21.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:21.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:21.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:21.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:21.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:21.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:21.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:21.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:21.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:21.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:21.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:21.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:21.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:20 smithi139 ceph-mon[91894]: from='client.? 172.21.15.62:0/3860979242' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:21.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:21.072 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:21.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:21.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:21.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:21.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:21.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:21.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:21.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:21.073 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:20 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:21.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:32:20] "GET /metrics HTTP/1.1" 200 38708 "" "Prometheus/2.33.4" 2024-03-24T11:32:21.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:21.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:21.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:21.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[90553]: from='client.64795 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:32:21.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:21.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:21.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:21.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[90553]: from='client.64801 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:32:21.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[90553]: pgmap v336: 161 pgs: 161 active+clean; 585 KiB data, 273 MiB used, 715 GiB / 715 GiB avail; 92 KiB/s rd, 170 B/s wr, 140 op/s 2024-03-24T11:32:21.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:21.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:21.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:21.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:21.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:21.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:21.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:21.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:21.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:21.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:21.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:21.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:21.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:21.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:21.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:21.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:21.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:21.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:21.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:21.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:21.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:21.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:21.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:21.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:21.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:21.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:21.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:21.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[90553]: from='client.? 172.21.15.62:0/3860979242' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:21.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:21.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:21.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:21.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:21.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:21.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:21.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:21.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:21.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:21.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:21.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:21.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:21.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:21.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[98643]: from='client.64795 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:32:21.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:21.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:21.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:21.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[98643]: from='client.64801 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:32:21.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[98643]: pgmap v336: 161 pgs: 161 active+clean; 585 KiB data, 273 MiB used, 715 GiB / 715 GiB avail; 92 KiB/s rd, 170 B/s wr, 140 op/s 2024-03-24T11:32:21.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:21.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:21.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:21.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:21.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:21.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:21.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:21.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:21.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:21.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:21.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:21.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:21.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:21.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:21.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:21.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:21.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:21.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:21.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:21.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:21.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:21.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:21.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:21.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:21.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:21.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:21.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:21.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[98643]: from='client.? 172.21.15.62:0/3860979242' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:21.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:21.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:21.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:21.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:21.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:21.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:21.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:21.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:21.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:21.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:20 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:21.310 INFO:teuthology.orchestra.run.smithi062.stdout:HEALTH_OK 2024-03-24T11:32:22.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='client.44838 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:32:22.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:22.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:22.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:22.000 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:22.001 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:22.001 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:22.001 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='client.64819 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:32:22.001 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:22.001 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:22.001 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:22.001 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:22.001 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:22.001 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:22.001 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:22.001 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:22.001 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:22.001 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:22.001 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:22.001 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:22.002 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:22.002 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:22.002 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:22.002 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:22.002 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:22.002 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:22.002 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:22.002 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:22.002 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:22.002 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:22.003 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='client.? 172.21.15.62:0/3008267434' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:32:22.003 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:22.003 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:22.003 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:22.003 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:22.003 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:22.003 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:22.003 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:22.003 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:22.003 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:22.003 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:22.003 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:22.004 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:22.004 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:22.004 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:22.004 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:22.004 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:22.004 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:22.004 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:22.004 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:22.004 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:22.004 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:22.004 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:22.004 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:22.004 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:22.004 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:22.005 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:22.005 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:22.005 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:22.005 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:22.005 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:22.005 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:22.005 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:22.005 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:22.005 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:22.005 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:21 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:22.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='client.44838 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:32:22.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:22.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:22.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:22.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:22.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:22.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:22.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='client.64819 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:32:22.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:22.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:22.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:22.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:22.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:22.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:22.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:22.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:22.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:22.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:22.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:22.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:22.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:22.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:22.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:22.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:22.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:22.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:22.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:22.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:22.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:22.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:22.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='client.44838 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:32:22.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:22.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:22.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:22.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:22.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:22.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:22.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='client.64819 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:32:22.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:22.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:22.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:22.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:22.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:22.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:22.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:22.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:22.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:22.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:22.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:22.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:22.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:22.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:22.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:22.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:22.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:22.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:22.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:22.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:22.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:22.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:22.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='client.? 172.21.15.62:0/3008267434' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:32:22.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:22.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:22.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:22.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:22.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:22.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:22.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:22.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:22.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:22.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:22.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:22.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:22.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:22.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:22.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:22.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:22.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:22.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:22.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:22.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:22.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:22.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:22.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:22.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:22.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:22.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:22.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:22.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='client.? 172.21.15.62:0/3008267434' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:32:22.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:22.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:22.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:22.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:22.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:22.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:22.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:22.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:22.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:22.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:22.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:22.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:22.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:22.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:22.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:22.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:22.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:22.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:22.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:22.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:22.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:22.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:22.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:22.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:22.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:22.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:22.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:22.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:22.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:22.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:22.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:22.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:22.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:22.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:22.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:22.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:22.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:22.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:22.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:22.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:22.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:22.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:22.238 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:21 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:23.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:23.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:23.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:23.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:23.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:23.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:23.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:23.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:23.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: pgmap v337: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 77 KiB/s rd, 85 B/s wr, 118 op/s 2024-03-24T11:32:23.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:23.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:23.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:23.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:23.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:23.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:23.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:23.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:23.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:23.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:23.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:23.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:23.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:23.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:23.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:23.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:23.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:23.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:23.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:23.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:23.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:23.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:23.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:23.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:23.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:23.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:23.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:23.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:23.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:23.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: pgmap v337: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 77 KiB/s rd, 85 B/s wr, 118 op/s 2024-03-24T11:32:23.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:23.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:23.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:23.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:23.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:23.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:23.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:23.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:23.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:23.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:23.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:23.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:23.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:23.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:23.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:23.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:23.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:23.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:23.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:23.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:23.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:23.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:23.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:23.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:23.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:23.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:23.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:23.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:23.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:23.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:23.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:23.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:23.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:23.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:23.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:23.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:23.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:23.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:23.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:23.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:23.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:23.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:23.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:23.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:23.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:23.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:23.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:23.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:23.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:23.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:23.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:23.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:23.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:23.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:23.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:23.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:23.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:23.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:23.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:23.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:23.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:23.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:23.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:23.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:23.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:23.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:23.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:23.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:23.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:23.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:23.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:23.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:23.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:23.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:23.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:22 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:23.236 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:23.237 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:23.237 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:23.237 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:23.237 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:23.237 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:23.237 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:23.237 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:23.237 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: pgmap v337: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 77 KiB/s rd, 85 B/s wr, 118 op/s 2024-03-24T11:32:23.237 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:23.237 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:23.237 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:23.237 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:23.237 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:23.237 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:23.238 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:23.238 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:23.238 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:23.238 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:23.238 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:23.238 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:23.238 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:23.238 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:23.238 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:23.238 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:23.238 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:23.238 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:23.238 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:23.238 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:23.239 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:23.239 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:23.239 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:23.239 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:23.239 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:23.239 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:23.239 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:23.239 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:23.239 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:23.239 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:23.239 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:23.239 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:23.239 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:23.239 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:23.239 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:23.240 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:23.240 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:23.240 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:23.240 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:23.240 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:23.240 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:23.240 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:23.240 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:23.240 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:23.240 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:23.240 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:23.240 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:23.240 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:22 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:24.198 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:24.198 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:24.198 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:24.198 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:24.198 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:32:24.198 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:24.198 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:24.198 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:24.198 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:24.198 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:24.198 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:24.199 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:24.199 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:24.199 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:24.199 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:24.199 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:24.199 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:24.199 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:24.199 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:24.199 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:24.199 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:24.199 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:24.199 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:24.199 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:24.200 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:24.200 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:24.200 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:24.200 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:24.200 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:24.200 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:24.200 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:24.200 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:24.200 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:24.200 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:24.200 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:24.200 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:24.200 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:24.200 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:24.200 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:24.201 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:24.201 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:24.201 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:24.201 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:24.201 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:24.201 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:24.201 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:24.201 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:24.201 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:24.201 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:24.201 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:24.201 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:24.201 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:24.201 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:24.202 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:24.202 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:24.202 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:24.202 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:24.202 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:24.202 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:24.202 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:24.202 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:24.202 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:24.202 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:24.202 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:24.202 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:24.202 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:23 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:24.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:24.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:24.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:24.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:24.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:32:24.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:24.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:24.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:24.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:24.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:24.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:24.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:24.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:24.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:24.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:24.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:24.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:24.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:24.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:24.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:24.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:24.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:24.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:24.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:24.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:24.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:24.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:24.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:24.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:24.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:24.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:24.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:24.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:24.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:24.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:32:24.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:24.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:24.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:24.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:24.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:24.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:24.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:24.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:24.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:24.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:24.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:24.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:24.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:24.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:24.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:24.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:24.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:24.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:24.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:24.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:24.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:24.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:24.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:24.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:24.231 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:24.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:24.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:24.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:24.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:24.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:24.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:24.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:24.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:24.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:24.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:24.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:24.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:24.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:24.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:24.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:24.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:24.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:24.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:24.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:24.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:24.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:24.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:24.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:24.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:24.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:24.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:24.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:24.233 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:24.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:24.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:24.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:24.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:24.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:24.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:24.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:24.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:24.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:24.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:24.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:24.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:24.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:24.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:24.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:24.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:24.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:24.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:24.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:24.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:24.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:24.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:24.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:24.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:24.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:24.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:24.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:24.235 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:24.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:24.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:24.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:24.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:24.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:24.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:24.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:24.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:24.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:24.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:24.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:24.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:24.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:24.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:24.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:24.237 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:23 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:25.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:25.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:25.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:25.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:25.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:25.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:25.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:25.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:25.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: pgmap v338: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 59 KiB/s rd, 85 B/s wr, 89 op/s 2024-03-24T11:32:25.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:25.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:25.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:25.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:25.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:25.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:25.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:25.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:25.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:25.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:25.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:25.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:25.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:25.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:25.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:25.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:25.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:25.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:25.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:25.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:25.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:25.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:25.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:25.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:25.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:25.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:25.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:25.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:25.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:25.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:25.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: pgmap v338: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 59 KiB/s rd, 85 B/s wr, 89 op/s 2024-03-24T11:32:25.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:25.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:25.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:25.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:25.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:25.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:25.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:25.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:25.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:25.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:25.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:25.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:25.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:25.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:25.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:25.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:25.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:25.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:25.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:25.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:25.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:25.232 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:25.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:25.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:25.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:25.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:25.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:25.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:25.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:25.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:25.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:25.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:25.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:25.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:25.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:25.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:25.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:25.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:25.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:25.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:25.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:25.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:25.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:25.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:25.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:25.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:25.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:25.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:25.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:25.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:25.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:25.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:25.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:25.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:25.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:25.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:25.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:25.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:25.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:25.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:25.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:25.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:25.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:25.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:25.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:25.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:25.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:25.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:25.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:25.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:25.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:25.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:25.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:25.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:25.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:25.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:25.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:25.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:25.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:25.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:25.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:25.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:25.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:25.236 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:25.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:25.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:25.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:25.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:25.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:25.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:25.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:25.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:24 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:25.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:25.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:25.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:25.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:25.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:25.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:25.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:25.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:25.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: pgmap v338: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 59 KiB/s rd, 85 B/s wr, 89 op/s 2024-03-24T11:32:25.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:25.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:25.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:25.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:25.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:25.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:25.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:25.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:25.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:25.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:25.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:25.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:25.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:25.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:25.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:25.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:25.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:25.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:25.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:25.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:25.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:25.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:25.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:25.360 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:25.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:25.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:25.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:25.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:25.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:25.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:25.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:25.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:25.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:25.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:25.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:25.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:25.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:25.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:25.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:25.361 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:25.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:25.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:25.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:25.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:25.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:25.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:25.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:25.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:25.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:25.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:25.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:25.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:25.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:25.362 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:25.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:25.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:25.363 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:24 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:26.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:26.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:26.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:26.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:26.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:26.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:26.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:26.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:26.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:26.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:26.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:26.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:26.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:26.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:26.109 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:26.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:26.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:26.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:26.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:26.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:26.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:26.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:26.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:26.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:26.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:26.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:26.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:26.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:26.110 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:26.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:26.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:26.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:26.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:26.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:26.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:26.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:26.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:26.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:26.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:26.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:26.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:26.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:26.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:26.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:26.111 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:26.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:26.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:26.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:26.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:26.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:26.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:26.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:26.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:26.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:26.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:26.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:26.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:26.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:26.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:26.112 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:26.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:26.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:26.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:26.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:26.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:26.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:26.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:26.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:26.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:26.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:26.113 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:25 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:26.202 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:26.202 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:26.202 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:26.203 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:26.203 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:26.203 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:26.203 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:26.203 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:26.203 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:26.203 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:26.203 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:26.203 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:26.203 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:26.203 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:26.203 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:26.203 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:26.203 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:26.203 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:26.204 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:26.204 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:26.204 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:26.204 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:26.204 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:26.204 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:26.204 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:26.204 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:26.204 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:26.204 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:26.204 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:26.204 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:26.204 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:26.205 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:26.205 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:26.205 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:26.205 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:26.205 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:26.205 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:26.205 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:26.205 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:26.205 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:26.205 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:26.205 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:26.206 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:26.206 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:26.206 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:26.206 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:26.206 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:26.206 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:26.206 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:26.206 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:26.206 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:26.206 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:26.206 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:26.206 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:26.206 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:26.206 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:26.206 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:26.207 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:26.207 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:26.207 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:26.207 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:26.207 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:26.207 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:26.207 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:26.207 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:26.207 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:26.207 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:26.207 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:26.207 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:26.207 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:26.208 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:26.208 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:26.208 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:26.208 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:26.208 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:26.208 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:26.208 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:26.208 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:26.208 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:26.208 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:26.208 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:26.208 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:26.208 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:26.208 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:26.208 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:26.208 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:26.209 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:26.209 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:26.209 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:26.209 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:26.209 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:26.209 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:26.209 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:26.209 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:26.209 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:26.209 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:26.209 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:26.209 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:26.209 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:26.210 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:26.210 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:26.210 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:26.210 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:26.210 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:26.210 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:26.210 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:26.210 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:26.210 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:26.210 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:26.210 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:26.210 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:26.210 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:26.210 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:26.210 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:26.210 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:26.211 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:26.211 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:26.211 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:26.211 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:26.211 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:26.211 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:26.211 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:26.211 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:26.211 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:26.211 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:26.211 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:26.211 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:26.211 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:26.212 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:26.212 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:26.212 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:26.212 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:26.212 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:26.212 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:26.212 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:26.212 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:26.212 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:26.212 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:26.212 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:26.212 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:26.212 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:26.213 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:25 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:27.205 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:27.205 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:27.206 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:27.206 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:27.206 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:27.206 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:27.206 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:27.206 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:27.206 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: pgmap v339: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 57 KiB/s rd, 85 B/s wr, 87 op/s 2024-03-24T11:32:27.206 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:27.206 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:27.206 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:27.206 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:27.206 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:27.206 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:27.207 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:27.207 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:27.207 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:27.207 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:27.207 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:27.207 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:27.207 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:27.207 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:32:27.207 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:27.207 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:27.207 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:27.207 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:27.207 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:27.207 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:27.207 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:27.208 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:27.208 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:27.208 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:27.208 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:27.208 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:27.208 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:27.208 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:27.208 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:27.208 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: pgmap v339: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 57 KiB/s rd, 85 B/s wr, 87 op/s 2024-03-24T11:32:27.208 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:27.208 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:27.208 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:27.209 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:27.209 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:27.209 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:27.209 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:27.209 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:27.209 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:27.209 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:27.209 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:27.209 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:27.209 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:27.209 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:32:27.209 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:27.209 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:27.209 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:27.209 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:27.209 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:27.210 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:27.210 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:27.210 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:27.210 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:27.210 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:27.210 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:27.210 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:27.210 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:27.210 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:27.210 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:27.210 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:27.210 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:27.210 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:27.210 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:27.211 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:27.211 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:27.211 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:27.211 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:27.211 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:27.211 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:27.211 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:27.211 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:27.211 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:27.211 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:27.211 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:27.211 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:27.211 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:27.211 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:27.211 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:27.212 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:27.212 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:27.212 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:27.212 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:27.212 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:27.212 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:27.212 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:27.212 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:27.212 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:27.212 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:27.212 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:27.212 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:27.212 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:27.212 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:27.213 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:27.213 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:27.213 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:27.213 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:27.213 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:27.213 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:27.213 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:27.213 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:27.213 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:27.213 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:27.213 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:27.213 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:27.213 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:27.213 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:27.213 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:26 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:27.216 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:27.216 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:27.216 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:27.216 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:27.216 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:27.216 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:27.216 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:27.216 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:27.216 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: pgmap v339: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 57 KiB/s rd, 85 B/s wr, 87 op/s 2024-03-24T11:32:27.217 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:27.217 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:27.217 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:27.217 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:27.217 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:27.217 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:27.217 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:27.217 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:27.217 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:27.217 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:27.217 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:27.217 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:27.217 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:27.217 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:32:27.217 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:27.218 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:27.218 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:27.218 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:27.218 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:27.218 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:27.218 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:27.218 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:27.218 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:27.218 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:27.218 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:27.219 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:27.219 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:27.219 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:27.219 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:27.219 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:27.219 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:27.219 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:27.219 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:27.219 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:27.219 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:27.219 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:27.219 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:27.219 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:27.219 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:27.219 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:27.219 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:27.220 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:27.220 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:27.220 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:27.220 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:27.220 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:27.220 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:27.220 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:27.220 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:26 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:28.180 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:28.180 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:28.180 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:28.180 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:28.180 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:28.180 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:28.180 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:28.181 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:28.181 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:28.181 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:28.181 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:28.181 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:28.181 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:28.181 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:28.181 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:28.181 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:28.181 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:28.181 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:28.181 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:28.181 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:28.181 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:28.182 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:28.182 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:28.182 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:28.182 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:28.182 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:28.182 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:28.182 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:28.182 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:28.182 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:28.182 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:28.182 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:28.182 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:28.182 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:28.182 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:28.182 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:28.183 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:28.183 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:28.183 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:28.183 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:28.183 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:28.183 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:28.183 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:28.183 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:28.183 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:28.183 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:28.183 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:28.183 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:28.183 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:28.184 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:28.184 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:28.184 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:28.184 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:28.184 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:28.184 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:28.184 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:28.184 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:28.184 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:28.184 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:28.185 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:28.185 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:28.185 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:28.185 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:28.185 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:28.185 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:28.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:28.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:28.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:28.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:28.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:28.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:28.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:28.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:28.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:28.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:28.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:28.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:28.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:28.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:28.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:28.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:28.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:28.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:28.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:28.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:28.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:28.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:28.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:28.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:28.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:28.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:28.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:28.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:28.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:28.229 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:28.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:28.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:28.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:28.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:28.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:28.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:28.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:28.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:28.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:28.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:28.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:28.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:28.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:28.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:28.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:28.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:28.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:28.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:28.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:28.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:28.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:28.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:28.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:28.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:28.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:28.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:28.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:28.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:28.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:28.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:28.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:28.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:28.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:28.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:28.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:28.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:28.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:28.232 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:28.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:28.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:28.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:28.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:28.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:28.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:28.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:28.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:28.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:28.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:28.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:28.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:28.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:28.233 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:28.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:28.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:28.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:28.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:28.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:28.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:28.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:28.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:28.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:28.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:28.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:28.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:28.234 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:28.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:28.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:28.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:28.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:28.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:28.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:28.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:28.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:28.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:28.235 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:28.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:28.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:28.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:28.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:28.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:28.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:28.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:28.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:28.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:28.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:28.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:28.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:28.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:28.236 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:28.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:28.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:28.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:28.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:28.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:28.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:28.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:28.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:28.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:28.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:28.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:29.128 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:29.128 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:29.128 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:29.128 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:29.128 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:29.128 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:29.128 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:29.128 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:29.128 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: pgmap v340: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 57 KiB/s rd, 85 B/s wr, 86 op/s 2024-03-24T11:32:29.129 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:29.129 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:29.129 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:29.129 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:29.129 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:29.129 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:29.129 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:29.129 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:29.129 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:29.129 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:29.129 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:29.129 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:29.129 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:29.130 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:29.130 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:29.130 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:29.130 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:29.130 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:29.130 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:29.130 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:29.130 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:29.130 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:29.130 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:29.130 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:29.130 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:29.130 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:29.130 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:29.131 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:29.131 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw.foo.smithi062.sacpds"}]: dispatch 2024-03-24T11:32:29.131 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw.foo.smithi062.sacpds"}]': finished 2024-03-24T11:32:29.131 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw.foo.smithi139.yljesh"}]: dispatch 2024-03-24T11:32:29.131 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw.foo.smithi139.yljesh"}]': finished 2024-03-24T11:32:29.131 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw.smpl.smithi062.xvpxnk"}]: dispatch 2024-03-24T11:32:29.131 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw.smpl.smithi062.xvpxnk"}]': finished 2024-03-24T11:32:29.131 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw.smpl.smithi139.pbtrmp"}]: dispatch 2024-03-24T11:32:29.131 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw.smpl.smithi139.pbtrmp"}]': finished 2024-03-24T11:32:29.131 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:29.131 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:29.131 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:29.131 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:29.131 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:29.131 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:29.132 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:29.132 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:28 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:29.132 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:29.132 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:29.132 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:29.132 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:29.132 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:29.132 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:29.132 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:29.133 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:29.133 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: pgmap v340: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 57 KiB/s rd, 85 B/s wr, 86 op/s 2024-03-24T11:32:29.133 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:29.133 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:29.133 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:29.133 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:29.133 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:29.133 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:29.133 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:29.133 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:29.133 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:29.133 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:29.133 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:29.133 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:29.133 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:29.133 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:29.134 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:29.134 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:29.134 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:29.134 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:29.134 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:29.134 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:29.134 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:29.134 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:29.134 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:29.134 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:29.134 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:29.135 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:29.135 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:29.135 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:29.135 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:29.135 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:29.135 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: pgmap v340: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 57 KiB/s rd, 85 B/s wr, 86 op/s 2024-03-24T11:32:29.135 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:29.135 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:29.135 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:29.135 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:29.135 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:29.135 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:29.135 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:29.135 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:29.136 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:29.136 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:29.136 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:29.136 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:29.136 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:29.136 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:29.136 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:29.136 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:29.136 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:29.136 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:29.136 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:29.136 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:29.136 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T11:32:29.136 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:29.137 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:29.137 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:29.137 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:29.137 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:29.137 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:29.137 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:29.137 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw.foo.smithi062.sacpds"}]: dispatch 2024-03-24T11:32:29.137 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw.foo.smithi062.sacpds"}]': finished 2024-03-24T11:32:29.137 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw.foo.smithi139.yljesh"}]: dispatch 2024-03-24T11:32:29.137 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw.foo.smithi139.yljesh"}]': finished 2024-03-24T11:32:29.137 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw.smpl.smithi062.xvpxnk"}]: dispatch 2024-03-24T11:32:29.137 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw.smpl.smithi062.xvpxnk"}]': finished 2024-03-24T11:32:29.137 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw.smpl.smithi139.pbtrmp"}]: dispatch 2024-03-24T11:32:29.138 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw.smpl.smithi139.pbtrmp"}]': finished 2024-03-24T11:32:29.138 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:29.138 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:29.138 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:29.138 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:29.138 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:29.138 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:29.138 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:29.138 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:29.138 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:29.138 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:29.138 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:29.139 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:29.139 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:29.139 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:29.139 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw.foo.smithi062.sacpds"}]: dispatch 2024-03-24T11:32:29.139 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw.foo.smithi062.sacpds"}]': finished 2024-03-24T11:32:29.139 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw.foo.smithi139.yljesh"}]: dispatch 2024-03-24T11:32:29.139 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw.foo.smithi139.yljesh"}]': finished 2024-03-24T11:32:29.139 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw.smpl.smithi062.xvpxnk"}]: dispatch 2024-03-24T11:32:29.139 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw.smpl.smithi062.xvpxnk"}]': finished 2024-03-24T11:32:29.139 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw.smpl.smithi139.pbtrmp"}]: dispatch 2024-03-24T11:32:29.139 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw.smpl.smithi139.pbtrmp"}]': finished 2024-03-24T11:32:29.139 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:29.139 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:29.139 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:29.139 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:29.140 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:29.140 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:29.140 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:29.140 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:28 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:30.055 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:29 smithi139 ceph-mon[91894]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:30.056 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:29 smithi139 ceph-mon[91894]: Upgrade: Setting container_image for all rgw 2024-03-24T11:32:30.056 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:29 smithi139 ceph-mon[91894]: Upgrade: Setting container_image for all rbd-mirror 2024-03-24T11:32:30.056 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:29 smithi139 ceph-mon[91894]: Upgrade: Setting container_image for all ceph-exporter 2024-03-24T11:32:30.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:29 smithi062 ceph-mon[90553]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:30.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:29 smithi062 ceph-mon[90553]: Upgrade: Setting container_image for all rgw 2024-03-24T11:32:30.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:29 smithi062 ceph-mon[90553]: Upgrade: Setting container_image for all rbd-mirror 2024-03-24T11:32:30.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:29 smithi062 ceph-mon[90553]: Upgrade: Setting container_image for all ceph-exporter 2024-03-24T11:32:30.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:29 smithi062 ceph-mon[98643]: Metadata not up to date on all hosts. Skipping non agent specs 2024-03-24T11:32:30.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:29 smithi062 ceph-mon[98643]: Upgrade: Setting container_image for all rgw 2024-03-24T11:32:30.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:29 smithi062 ceph-mon[98643]: Upgrade: Setting container_image for all rbd-mirror 2024-03-24T11:32:30.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:29 smithi062 ceph-mon[98643]: Upgrade: Setting container_image for all ceph-exporter 2024-03-24T11:32:31.096 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:32:30 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:32:30] "GET /metrics HTTP/1.1" 200 38754 "" "Prometheus/2.33.4" 2024-03-24T11:32:31.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:30 smithi062 ceph-mon[90553]: pgmap v341: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 57 KiB/s rd, 85 B/s wr, 87 op/s 2024-03-24T11:32:31.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:31.096 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi062.dggsvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-03-24T11:32:31.097 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:31.097 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:30 smithi062 ceph-mon[98643]: pgmap v341: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 57 KiB/s rd, 85 B/s wr, 87 op/s 2024-03-24T11:32:31.097 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:31.097 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi062.dggsvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-03-24T11:32:31.097 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:31.180 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:30 smithi139 ceph-mon[91894]: pgmap v341: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 57 KiB/s rd, 85 B/s wr, 87 op/s 2024-03-24T11:32:31.181 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:31.181 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi062.dggsvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-03-24T11:32:31.181 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:32.194 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:31 smithi062 ceph-mon[90553]: Upgrade: Updating iscsi.foo.smithi062.dggsvk 2024-03-24T11:32:32.194 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:31 smithi062 ceph-mon[90553]: Deploying daemon iscsi.foo.smithi062.dggsvk on smithi062 2024-03-24T11:32:32.194 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:32.195 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:32.195 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:31 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:32.195 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:31 smithi062 ceph-mon[98643]: Upgrade: Updating iscsi.foo.smithi062.dggsvk 2024-03-24T11:32:32.195 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:31 smithi062 ceph-mon[98643]: Deploying daemon iscsi.foo.smithi062.dggsvk on smithi062 2024-03-24T11:32:32.195 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:32.195 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:32.195 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:31 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:32.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:31 smithi139 ceph-mon[91894]: Upgrade: Updating iscsi.foo.smithi062.dggsvk 2024-03-24T11:32:32.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:31 smithi139 ceph-mon[91894]: Deploying daemon iscsi.foo.smithi062.dggsvk on smithi062 2024-03-24T11:32:32.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:32.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:32.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:31 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:33.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:32 smithi139 ceph-mon[91894]: pgmap v342: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 21 KiB/s rd, 0 B/s wr, 30 op/s 2024-03-24T11:32:33.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:32 smithi062 ceph-mon[90553]: pgmap v342: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 21 KiB/s rd, 0 B/s wr, 30 op/s 2024-03-24T11:32:33.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:32 smithi062 ceph-mon[98643]: pgmap v342: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 21 KiB/s rd, 0 B/s wr, 30 op/s 2024-03-24T11:32:34.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:33 smithi062 ceph-mon[98643]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:32:34.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:33 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:32:34.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:33 smithi139 ceph-mon[91894]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:32:35.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:34 smithi062 ceph-mon[98643]: pgmap v343: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:32:35.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:34 smithi062 ceph-mon[90553]: pgmap v343: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:32:35.259 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:34 smithi139 ceph-mon[91894]: pgmap v343: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:32:37.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:36 smithi062 ceph-mon[90553]: pgmap v344: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:32:37.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:36 smithi062 ceph-mon[98643]: pgmap v344: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:32:37.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:36 smithi139 ceph-mon[91894]: pgmap v344: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:32:39.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:38 smithi062 ceph-mon[90553]: pgmap v345: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:32:39.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:38 smithi062 ceph-mon[98643]: pgmap v345: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:32:39.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:38 smithi139 ceph-mon[91894]: pgmap v345: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:32:41.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:40 smithi062 ceph-mon[90553]: pgmap v346: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:32:41.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:32:40 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:32:40] "GET /metrics HTTP/1.1" 200 38754 "" "Prometheus/2.33.4" 2024-03-24T11:32:41.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:40 smithi062 ceph-mon[98643]: pgmap v346: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:32:41.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:40 smithi139 ceph-mon[91894]: pgmap v346: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:32:42.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:41 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:32:42.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:41 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:32:42.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:41 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:32:43.075 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:42 smithi062 ceph-mon[90553]: pgmap v347: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:32:43.075 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:42 smithi062 ceph-mon[98643]: pgmap v347: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:32:43.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:42 smithi139 ceph-mon[91894]: pgmap v347: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:32:44.230 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:43 smithi062 ceph-mon[90553]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:32:44.230 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:43 smithi062 ceph-mon[98643]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:32:44.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:43 smithi139 ceph-mon[91894]: from='client.15486 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:32:44.726 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:32:44 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:32:44.379+0000 7f4e1b26f700 -1 log_channel(cephadm) log [ERR] : Upgrade: Paused due to UPGRADE_REDEPLOY_DAEMON: Upgrading daemon iscsi.foo.smithi062.dggsvk on host smithi062 failed. 2024-03-24T11:32:45.013 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:44 smithi062 ceph-mon[90553]: pgmap v348: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:32:45.014 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:45.014 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:45.014 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:45.014 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:45.014 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:45.014 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:45.014 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:44 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:45.014 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:44 smithi062 ceph-mon[98643]: pgmap v348: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:32:45.014 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:45.014 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:45.014 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:45.015 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:45.015 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:45.015 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:45.015 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:44 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:45.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:44 smithi139 ceph-mon[91894]: pgmap v348: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:32:45.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:45.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:45.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:45.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:45.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:45.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:45.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:44 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:46.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:45 smithi062 ceph-mon[90553]: Upgrade: Paused due to UPGRADE_REDEPLOY_DAEMON: Upgrading daemon iscsi.foo.smithi062.dggsvk on host smithi062 failed. 2024-03-24T11:32:46.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:45 smithi062 ceph-mon[90553]: pgmap v349: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T11:32:46.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:45 smithi062 ceph-mon[90553]: Health check failed: Upgrading daemon iscsi.foo.smithi062.dggsvk on host smithi062 failed. (UPGRADE_REDEPLOY_DAEMON) 2024-03-24T11:32:46.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:45 smithi062 ceph-mon[98643]: Upgrade: Paused due to UPGRADE_REDEPLOY_DAEMON: Upgrading daemon iscsi.foo.smithi062.dggsvk on host smithi062 failed. 2024-03-24T11:32:46.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:45 smithi062 ceph-mon[98643]: pgmap v349: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T11:32:46.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:45 smithi062 ceph-mon[98643]: Health check failed: Upgrading daemon iscsi.foo.smithi062.dggsvk on host smithi062 failed. (UPGRADE_REDEPLOY_DAEMON) 2024-03-24T11:32:46.296 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:45 smithi139 ceph-mon[91894]: Upgrade: Paused due to UPGRADE_REDEPLOY_DAEMON: Upgrading daemon iscsi.foo.smithi062.dggsvk on host smithi062 failed. 2024-03-24T11:32:46.297 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:45 smithi139 ceph-mon[91894]: pgmap v349: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T11:32:46.297 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:45 smithi139 ceph-mon[91894]: Health check failed: Upgrading daemon iscsi.foo.smithi062.dggsvk on host smithi062 failed. (UPGRADE_REDEPLOY_DAEMON) 2024-03-24T11:32:48.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:47 smithi062 ceph-mon[90553]: pgmap v350: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 866 B/s rd, 0 op/s 2024-03-24T11:32:48.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:47 smithi062 ceph-mon[98643]: pgmap v350: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 866 B/s rd, 0 op/s 2024-03-24T11:32:48.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:47 smithi139 ceph-mon[91894]: pgmap v350: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 866 B/s rd, 0 op/s 2024-03-24T11:32:50.260 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:49 smithi062 ceph-mon[98643]: pgmap v351: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 866 B/s rd, 0 op/s 2024-03-24T11:32:50.260 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:50.261 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:50.261 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:49 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:50.261 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:49 smithi062 ceph-mon[90553]: pgmap v351: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 866 B/s rd, 0 op/s 2024-03-24T11:32:50.261 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:50.261 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:50.261 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:49 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:50.261 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:49 smithi139 ceph-mon[91894]: pgmap v351: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 866 B/s rd, 0 op/s 2024-03-24T11:32:50.261 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:50.262 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:50.262 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:49 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:50.977 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:32:50 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:32:50] "GET /metrics HTTP/1.1" 200 38771 "" "Prometheus/2.33.4" 2024-03-24T11:32:51.713 INFO:teuthology.orchestra.run.smithi062.stdout:true 2024-03-24T11:32:52.061 INFO:teuthology.orchestra.run.smithi062.stdout:"Error: UPGRADE_REDEPLOY_DAEMON: Upgrading daemon iscsi.foo.smithi062.dggsvk on host smithi062 failed." 2024-03-24T11:32:52.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:51 smithi139 ceph-mon[91894]: pgmap v352: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 385 B/s rd, 0 op/s 2024-03-24T11:32:52.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:51 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:52.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:51 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:52.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:51 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:52.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:51 smithi062 ceph-mon[90553]: pgmap v352: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 385 B/s rd, 0 op/s 2024-03-24T11:32:52.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:51 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:52.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:51 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:52.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:51 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:52.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:51 smithi062 ceph-mon[98643]: pgmap v352: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 385 B/s rd, 0 op/s 2024-03-24T11:32:52.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:51 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:52.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:51 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:52.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:51 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:52.927 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph orch ps' 2024-03-24T11:32:53.194 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:53 smithi062 ceph-mon[90553]: from='client.44862 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:32:53.194 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:53 smithi062 ceph-mon[90553]: from='client.64837 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:32:53.194 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:53 smithi062 ceph-mon[98643]: from='client.44862 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:32:53.195 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:53 smithi062 ceph-mon[98643]: from='client.64837 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:32:53.295 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:53 smithi139 ceph-mon[91894]: from='client.44862 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:32:53.295 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:53 smithi139 ceph-mon[91894]: from='client.64837 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:32:54.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:54 smithi062 ceph-mon[90553]: pgmap v353: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 385 B/s rd, 0 op/s 2024-03-24T11:32:54.357 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:54 smithi062 ceph-mon[98643]: pgmap v353: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 385 B/s rd, 0 op/s 2024-03-24T11:32:54.357 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:54 smithi139 ceph-mon[91894]: pgmap v353: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 385 B/s rd, 0 op/s 2024-03-24T11:32:55.617 INFO:teuthology.orchestra.run.smithi062.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T11:32:55.617 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi062 smithi062 running 4s ago 22m - - 2024-03-24T11:32:55.617 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi139 smithi139 running 6s ago 22m - - 2024-03-24T11:32:55.617 INFO:teuthology.orchestra.run.smithi062.stdout:alertmanager.a smithi062 *:9093,9094 running (12m) 4s ago 16m - - 0.23.0 ba2b418f427c 818298d58140 2024-03-24T11:32:55.617 INFO:teuthology.orchestra.run.smithi062.stdout:grafana.a smithi139 *:3000 running (11m) 6s ago 16m 55.0M - 8.3.5 dad864ee21e9 b48167e7171e 2024-03-24T11:32:55.618 INFO:teuthology.orchestra.run.smithi062.stdout:iscsi.foo.smithi062.dggsvk smithi062 running (10m) 4s ago 15m 66.2M - 3.5 e1d6a67b021e 0a98806ec839 2024-03-24T11:32:55.618 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.x smithi139 *:8443,9283,8765 running (9m) 6s ago 21m 453M - 19.0.0-1667-gdb0330b1 4f325f8440e9 d94987b11387 2024-03-24T11:32:55.618 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.y smithi062 *:8443,9283,8765 running (11m) 4s ago 23m - - 19.0.0-1667-gdb0330b1 4f325f8440e9 ca1a49028d65 2024-03-24T11:32:55.618 INFO:teuthology.orchestra.run.smithi062.stdout:mon.a smithi062 running (8m) 4s ago 23m - 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 f7afe6e28c88 2024-03-24T11:32:55.618 INFO:teuthology.orchestra.run.smithi062.stdout:mon.b smithi139 running (8m) 6s ago 21m 58.4M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 ca357beea4dd 2024-03-24T11:32:55.618 INFO:teuthology.orchestra.run.smithi062.stdout:mon.c smithi062 running (8m) 4s ago 22m - 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 459e7449ec39 2024-03-24T11:32:55.618 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.a smithi062 *:9100 running (12m) 4s ago 17m - - 1.3.1 1dbe0e931976 79d5a02a999f 2024-03-24T11:32:55.618 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.b smithi139 *:9100 running (11m) 6s ago 17m 15.8M - 1.3.1 1dbe0e931976 8f9f4608e465 2024-03-24T11:32:55.618 INFO:teuthology.orchestra.run.smithi062.stdout:osd.0 smithi062 running (6m) 4s ago 21m - 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 dfe4920b49c4 2024-03-24T11:32:55.618 INFO:teuthology.orchestra.run.smithi062.stdout:osd.1 smithi062 running (6m) 4s ago 20m - 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 c443b3fe97c1 2024-03-24T11:32:55.618 INFO:teuthology.orchestra.run.smithi062.stdout:osd.2 smithi062 running (4m) 4s ago 20m - 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 0eba40dc96b3 2024-03-24T11:32:55.618 INFO:teuthology.orchestra.run.smithi062.stdout:osd.3 smithi062 running (3m) 4s ago 20m - 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 7501d2e4c718 2024-03-24T11:32:55.618 INFO:teuthology.orchestra.run.smithi062.stdout:osd.4 smithi139 running (3m) 6s ago 19m 59.2M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 5d0f7f05ec6d 2024-03-24T11:32:55.619 INFO:teuthology.orchestra.run.smithi062.stdout:osd.5 smithi139 running (2m) 6s ago 19m 48.6M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 4ea95bfd2127 2024-03-24T11:32:55.619 INFO:teuthology.orchestra.run.smithi062.stdout:osd.6 smithi139 running (110s) 6s ago 18m 49.8M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 c67fd9426ac5 2024-03-24T11:32:55.619 INFO:teuthology.orchestra.run.smithi062.stdout:osd.7 smithi139 running (81s) 6s ago 18m 48.6M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 d4fbd9bccd09 2024-03-24T11:32:55.619 INFO:teuthology.orchestra.run.smithi062.stdout:prometheus.a smithi139 *:9095 running (10m) 6s ago 17m 71.3M - 2.33.4 514e6a882f6e 2ed47f810d33 2024-03-24T11:32:55.619 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi062.sacpds smithi062 *:8000 running (60s) 4s ago 16m - - 19.0.0-1667-gdb0330b1 4f325f8440e9 bdccb278a0c3 2024-03-24T11:32:55.619 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi139.yljesh smithi139 *:8000 running (48s) 6s ago 16m 75.5M - 19.0.0-1667-gdb0330b1 4f325f8440e9 9c325556e724 2024-03-24T11:32:55.619 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi062.xvpxnk smithi062 *:80 running (54s) 4s ago 15m - - 19.0.0-1667-gdb0330b1 4f325f8440e9 88bcf6d3dca0 2024-03-24T11:32:55.619 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi139.pbtrmp smithi139 *:80 running (39s) 6s ago 15m 77.2M - 19.0.0-1667-gdb0330b1 4f325f8440e9 c9bd2dd9ce30 2024-03-24T11:32:56.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:56 smithi139 ceph-mon[91894]: pgmap v354: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 385 B/s rd, 0 op/s 2024-03-24T11:32:56.373 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:56 smithi062 ceph-mon[90553]: pgmap v354: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 385 B/s rd, 0 op/s 2024-03-24T11:32:56.373 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:56 smithi062 ceph-mon[98643]: pgmap v354: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 385 B/s rd, 0 op/s 2024-03-24T11:32:56.412 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph versions' 2024-03-24T11:32:57.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:57 smithi139 ceph-mon[91894]: from='client.64843 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:32:57.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:57.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:57 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:32:57.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:57 smithi062 ceph-mon[90553]: from='client.64843 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:32:57.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:57.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:57 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:32:57.421 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:57 smithi062 ceph-mon[98643]: from='client.64843 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:32:57.421 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:57.422 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:57 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:32:58.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:58 smithi139 ceph-mon[91894]: pgmap v355: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:32:58.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:58 smithi139 ceph-mon[91894]: from='client.? 172.21.15.62:0/512972502' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-24T11:32:58.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:58 smithi139 ceph-mon[91894]: from='client.? 172.21.15.62:0/3130195591' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6801/1501055735"}]: dispatch 2024-03-24T11:32:58.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:58.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:58.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:58.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:58.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:58.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:58 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:58.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:58 smithi062 ceph-mon[90553]: pgmap v355: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:32:58.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:58 smithi062 ceph-mon[90553]: from='client.? 172.21.15.62:0/512972502' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-24T11:32:58.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:58 smithi062 ceph-mon[90553]: from='client.? 172.21.15.62:0/3130195591' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6801/1501055735"}]: dispatch 2024-03-24T11:32:58.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:58.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:58.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:58.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:58.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:58.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:58 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:58.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:58 smithi062 ceph-mon[98643]: pgmap v355: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:32:58.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:58 smithi062 ceph-mon[98643]: from='client.? 172.21.15.62:0/512972502' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-24T11:32:58.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:58 smithi062 ceph-mon[98643]: from='client.? 172.21.15.62:0/3130195591' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6801/1501055735"}]: dispatch 2024-03-24T11:32:58.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:58.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:58.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:58.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:32:58.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:32:58.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:58 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:32:58.538 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:32:58.538 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": { 2024-03-24T11:32:58.538 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 3 2024-03-24T11:32:58.538 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:32:58.538 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": { 2024-03-24T11:32:58.538 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-24T11:32:58.538 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:32:58.538 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": { 2024-03-24T11:32:58.539 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 8 2024-03-24T11:32:58.539 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:32:58.539 INFO:teuthology.orchestra.run.smithi062.stdout: "rgw": { 2024-03-24T11:32:58.539 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 4 2024-03-24T11:32:58.539 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:32:58.539 INFO:teuthology.orchestra.run.smithi062.stdout: "overall": { 2024-03-24T11:32:58.539 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 17 2024-03-24T11:32:58.539 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-24T11:32:58.539 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:32:59.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:59 smithi062 ceph-mon[90553]: pgmap v356: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:32:59.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:59 smithi062 ceph-mon[90553]: pgmap v357: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:32:59.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:59 smithi062 ceph-mon[90553]: pgmap v358: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:32:59.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:59 smithi062 ceph-mon[90553]: from='client.? 172.21.15.62:0/3130195591' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6801/1501055735"}]': finished 2024-03-24T11:32:59.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:59 smithi062 ceph-mon[90553]: osdmap e175: 8 total, 8 up, 8 in 2024-03-24T11:32:59.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:59 smithi062 ceph-mon[90553]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-03-24T11:32:59.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:59 smithi062 ceph-mon[90553]: from='client.? 172.21.15.62:0/1224058804' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6800/1501055735"}]: dispatch 2024-03-24T11:32:59.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:32:59 smithi062 ceph-mon[90553]: from='client.? 172.21.15.62:0/1183956866' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:59.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:59 smithi062 ceph-mon[98643]: pgmap v356: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:32:59.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:59 smithi062 ceph-mon[98643]: pgmap v357: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:32:59.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:59 smithi062 ceph-mon[98643]: pgmap v358: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:32:59.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:59 smithi062 ceph-mon[98643]: from='client.? 172.21.15.62:0/3130195591' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6801/1501055735"}]': finished 2024-03-24T11:32:59.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:59 smithi062 ceph-mon[98643]: osdmap e175: 8 total, 8 up, 8 in 2024-03-24T11:32:59.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:59 smithi062 ceph-mon[98643]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-03-24T11:32:59.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:59 smithi062 ceph-mon[98643]: from='client.? 172.21.15.62:0/1224058804' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6800/1501055735"}]: dispatch 2024-03-24T11:32:59.228 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:32:59 smithi062 ceph-mon[98643]: from='client.? 172.21.15.62:0/1183956866' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:59.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:59 smithi139 ceph-mon[91894]: pgmap v356: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:32:59.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:59 smithi139 ceph-mon[91894]: pgmap v357: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:32:59.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:59 smithi139 ceph-mon[91894]: pgmap v358: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:32:59.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:59 smithi139 ceph-mon[91894]: from='client.? 172.21.15.62:0/3130195591' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6801/1501055735"}]': finished 2024-03-24T11:32:59.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:59 smithi139 ceph-mon[91894]: osdmap e175: 8 total, 8 up, 8 in 2024-03-24T11:32:59.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:59 smithi139 ceph-mon[91894]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-03-24T11:32:59.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:59 smithi139 ceph-mon[91894]: from='client.? 172.21.15.62:0/1224058804' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6800/1501055735"}]: dispatch 2024-03-24T11:32:59.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:32:59 smithi139 ceph-mon[91894]: from='client.? 172.21.15.62:0/1183956866' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:32:59.363 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-03-24T11:33:00.770 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:00 smithi062 ceph-mon[90553]: from='client.? 172.21.15.62:0/1224058804' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6800/1501055735"}]': finished 2024-03-24T11:33:00.771 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:00 smithi062 ceph-mon[90553]: osdmap e176: 8 total, 8 up, 8 in 2024-03-24T11:33:00.771 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:00 smithi062 ceph-mon[90553]: pgmap v361: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 986 B/s rd, 1 op/s 2024-03-24T11:33:00.771 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:00 smithi062 ceph-mon[90553]: from='client.? 172.21.15.62:0/3199075583' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/3159277232"}]: dispatch 2024-03-24T11:33:00.771 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:00 smithi062 ceph-mon[98643]: from='client.? 172.21.15.62:0/1224058804' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6800/1501055735"}]': finished 2024-03-24T11:33:00.771 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:00 smithi062 ceph-mon[98643]: osdmap e176: 8 total, 8 up, 8 in 2024-03-24T11:33:00.771 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:00 smithi062 ceph-mon[98643]: pgmap v361: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 986 B/s rd, 1 op/s 2024-03-24T11:33:00.771 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:00 smithi062 ceph-mon[98643]: from='client.? 172.21.15.62:0/3199075583' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/3159277232"}]: dispatch 2024-03-24T11:33:00.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:00 smithi139 ceph-mon[91894]: from='client.? 172.21.15.62:0/1224058804' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:6800/1501055735"}]': finished 2024-03-24T11:33:00.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:00 smithi139 ceph-mon[91894]: osdmap e176: 8 total, 8 up, 8 in 2024-03-24T11:33:00.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:00 smithi139 ceph-mon[91894]: pgmap v361: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 986 B/s rd, 1 op/s 2024-03-24T11:33:00.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:00 smithi139 ceph-mon[91894]: from='client.? 172.21.15.62:0/3199075583' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/3159277232"}]: dispatch 2024-03-24T11:33:01.028 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:33:00 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:33:00] "GET /metrics HTTP/1.1" 200 38845 "" "Prometheus/2.33.4" 2024-03-24T11:33:01.223 INFO:teuthology.orchestra.run.smithi062.stdout:wait for servicemap items w/ changing names to refresh 2024-03-24T11:33:01.777 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:01 smithi062 ceph-mon[90553]: from='client.? 172.21.15.62:0/3199075583' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/3159277232"}]': finished 2024-03-24T11:33:01.777 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:01 smithi062 ceph-mon[90553]: osdmap e177: 8 total, 8 up, 8 in 2024-03-24T11:33:01.777 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:01 smithi062 ceph-mon[90553]: from='client.? 172.21.15.62:0/125206990' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/3373411673"}]: dispatch 2024-03-24T11:33:01.777 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:01 smithi062 ceph-mon[98643]: from='client.? 172.21.15.62:0/3199075583' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/3159277232"}]': finished 2024-03-24T11:33:01.777 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:01 smithi062 ceph-mon[98643]: osdmap e177: 8 total, 8 up, 8 in 2024-03-24T11:33:01.777 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:01 smithi062 ceph-mon[98643]: from='client.? 172.21.15.62:0/125206990' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/3373411673"}]: dispatch 2024-03-24T11:33:01.800 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'sleep 60' 2024-03-24T11:33:01.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:01 smithi139 ceph-mon[91894]: from='client.? 172.21.15.62:0/3199075583' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/3159277232"}]': finished 2024-03-24T11:33:01.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:01 smithi139 ceph-mon[91894]: osdmap e177: 8 total, 8 up, 8 in 2024-03-24T11:33:01.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:01 smithi139 ceph-mon[91894]: from='client.? 172.21.15.62:0/125206990' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/3373411673"}]: dispatch 2024-03-24T11:33:02.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:02 smithi062 ceph-mon[98643]: from='client.? 172.21.15.62:0/125206990' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/3373411673"}]': finished 2024-03-24T11:33:02.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:02 smithi062 ceph-mon[98643]: osdmap e178: 8 total, 8 up, 8 in 2024-03-24T11:33:02.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:02 smithi062 ceph-mon[98643]: pgmap v364: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-24T11:33:02.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:02 smithi062 ceph-mon[98643]: from='client.? 172.21.15.62:0/622584573' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/1517778727"}]: dispatch 2024-03-24T11:33:02.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:02 smithi062 ceph-mon[98643]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/1517778727"}]: dispatch 2024-03-24T11:33:02.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:02 smithi062 ceph-mon[90553]: from='client.? 172.21.15.62:0/125206990' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/3373411673"}]': finished 2024-03-24T11:33:02.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:02 smithi062 ceph-mon[90553]: osdmap e178: 8 total, 8 up, 8 in 2024-03-24T11:33:02.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:02 smithi062 ceph-mon[90553]: pgmap v364: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-24T11:33:02.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:02 smithi062 ceph-mon[90553]: from='client.? 172.21.15.62:0/622584573' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/1517778727"}]: dispatch 2024-03-24T11:33:02.728 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:02 smithi062 ceph-mon[90553]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/1517778727"}]: dispatch 2024-03-24T11:33:02.791 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:02 smithi139 ceph-mon[91894]: from='client.? 172.21.15.62:0/125206990' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/3373411673"}]': finished 2024-03-24T11:33:02.791 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:02 smithi139 ceph-mon[91894]: osdmap e178: 8 total, 8 up, 8 in 2024-03-24T11:33:02.791 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:02 smithi139 ceph-mon[91894]: pgmap v364: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-24T11:33:02.791 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:02 smithi139 ceph-mon[91894]: from='client.? 172.21.15.62:0/622584573' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/1517778727"}]: dispatch 2024-03-24T11:33:02.791 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:02 smithi139 ceph-mon[91894]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/1517778727"}]: dispatch 2024-03-24T11:33:03.523 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:03 smithi062 ceph-mon[90553]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/1517778727"}]': finished 2024-03-24T11:33:03.523 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:03 smithi062 ceph-mon[90553]: osdmap e179: 8 total, 8 up, 8 in 2024-03-24T11:33:03.524 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:03 smithi062 ceph-mon[90553]: from='client.? 172.21.15.62:0/211185976' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/1216366974"}]: dispatch 2024-03-24T11:33:03.524 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:03 smithi062 ceph-mon[98643]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/1517778727"}]': finished 2024-03-24T11:33:03.524 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:03 smithi062 ceph-mon[98643]: osdmap e179: 8 total, 8 up, 8 in 2024-03-24T11:33:03.524 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:03 smithi062 ceph-mon[98643]: from='client.? 172.21.15.62:0/211185976' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/1216366974"}]: dispatch 2024-03-24T11:33:03.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:03 smithi139 ceph-mon[91894]: from='client.? ' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/1517778727"}]': finished 2024-03-24T11:33:03.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:03 smithi139 ceph-mon[91894]: osdmap e179: 8 total, 8 up, 8 in 2024-03-24T11:33:03.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:03 smithi139 ceph-mon[91894]: from='client.? 172.21.15.62:0/211185976' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/1216366974"}]: dispatch 2024-03-24T11:33:04.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:04 smithi139 ceph-mon[91894]: pgmap v366: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:33:04.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:04 smithi139 ceph-mon[91894]: from='client.? 172.21.15.62:0/211185976' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/1216366974"}]': finished 2024-03-24T11:33:04.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:04 smithi139 ceph-mon[91894]: osdmap e180: 8 total, 8 up, 8 in 2024-03-24T11:33:04.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:04 smithi062 ceph-mon[90553]: pgmap v366: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:33:04.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:04 smithi062 ceph-mon[90553]: from='client.? 172.21.15.62:0/211185976' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/1216366974"}]': finished 2024-03-24T11:33:04.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:04 smithi062 ceph-mon[90553]: osdmap e180: 8 total, 8 up, 8 in 2024-03-24T11:33:04.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:04 smithi062 ceph-mon[98643]: pgmap v366: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail 2024-03-24T11:33:04.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:04 smithi062 ceph-mon[98643]: from='client.? 172.21.15.62:0/211185976' entity='client.iscsi.foo.smithi062.dggsvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.62:0/1216366974"}]': finished 2024-03-24T11:33:04.978 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:04 smithi062 ceph-mon[98643]: osdmap e180: 8 total, 8 up, 8 in 2024-03-24T11:33:06.636 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:06 smithi062 ceph-mon[90553]: pgmap v368: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 1020 B/s rd, 0 op/s 2024-03-24T11:33:06.636 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:06 smithi062 ceph-mon[98643]: pgmap v368: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 1020 B/s rd, 0 op/s 2024-03-24T11:33:06.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:06 smithi139 ceph-mon[91894]: pgmap v368: 161 pgs: 161 active+clean; 585 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 1020 B/s rd, 0 op/s 2024-03-24T11:33:07.584 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:07 smithi062 ceph-mon[90553]: from='client.44892 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:33:07.585 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:07 smithi062 ceph-mon[98643]: from='client.44892 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:33:07.803 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:07 smithi139 ceph-mon[91894]: from='client.44892 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:33:08.719 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:08 smithi062 ceph-mon[90553]: pgmap v369: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 851 B/s rd, 0 op/s 2024-03-24T11:33:08.719 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:08 smithi062 ceph-mon[98643]: pgmap v369: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 851 B/s rd, 0 op/s 2024-03-24T11:33:08.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:08 smithi139 ceph-mon[91894]: pgmap v369: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 851 B/s rd, 0 op/s 2024-03-24T11:33:10.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:10.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:10.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:10 smithi062 ceph-mon[98643]: pgmap v370: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T11:33:10.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:10 smithi062 ceph-mon[98643]: pgmap v371: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-03-24T11:33:10.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:10.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:10.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:10.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:10 smithi062 ceph-mon[90553]: pgmap v370: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T11:33:10.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:10 smithi062 ceph-mon[90553]: pgmap v371: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-03-24T11:33:10.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:10.519 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:10.520 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:10.520 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:10 smithi139 ceph-mon[91894]: pgmap v370: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T11:33:10.520 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:10 smithi139 ceph-mon[91894]: pgmap v371: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-03-24T11:33:10.520 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:11.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:33:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:33:10] "GET /metrics HTTP/1.1" 200 38845 "" "Prometheus/2.33.4" 2024-03-24T11:33:12.261 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:12 smithi139 ceph-mon[91894]: pgmap v372: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T11:33:12.261 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:12.261 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:12 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:33:12.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:12 smithi062 ceph-mon[90553]: pgmap v372: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T11:33:12.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:12.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:12 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:33:12.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:12 smithi062 ceph-mon[98643]: pgmap v372: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T11:33:12.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:12.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:12 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:33:14.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:14 smithi062 ceph-mon[90553]: pgmap v373: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-24T11:33:14.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:14 smithi062 ceph-mon[98643]: pgmap v373: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-24T11:33:14.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:14 smithi139 ceph-mon[91894]: pgmap v373: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-24T11:33:16.430 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:16 smithi062 ceph-mon[98643]: pgmap v374: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-24T11:33:16.430 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:16 smithi062 ceph-mon[90553]: pgmap v374: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-24T11:33:16.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:16 smithi139 ceph-mon[91894]: pgmap v374: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-24T11:33:18.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:18 smithi139 ceph-mon[91894]: from='client.44892 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:33:18.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:18.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:18.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:18 smithi139 ceph-mon[91894]: pgmap v375: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-24T11:33:18.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:18.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:33:18.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:33:18.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:18 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:18.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:18 smithi062 ceph-mon[90553]: from='client.44892 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:33:18.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:18.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:18.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:18 smithi062 ceph-mon[90553]: pgmap v375: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-24T11:33:18.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:18.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:33:18.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:33:18.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:18 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:18.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:18 smithi062 ceph-mon[98643]: from='client.44892 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:33:18.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:18.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:18.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:18 smithi062 ceph-mon[98643]: pgmap v375: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-24T11:33:18.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:18.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:33:18.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:33:18.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:18 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:19.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:19 smithi139 ceph-mon[91894]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-03-24T11:33:19.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:19 smithi062 ceph-mon[90553]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-03-24T11:33:19.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:19 smithi062 ceph-mon[98643]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-03-24T11:33:20.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:20 smithi139 ceph-mon[91894]: pgmap v376: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-24T11:33:20.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:20 smithi062 ceph-mon[90553]: pgmap v376: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-24T11:33:20.478 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:20 smithi062 ceph-mon[98643]: pgmap v376: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-24T11:33:21.137 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:33:20 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:33:20] "GET /metrics HTTP/1.1" 200 38843 "" "Prometheus/2.33.4" 2024-03-24T11:33:21.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:21 smithi062 ceph-mon[90553]: pgmap v377: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 858 B/s rd, 0 op/s 2024-03-24T11:33:21.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:21 smithi062 ceph-mon[98643]: pgmap v377: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 858 B/s rd, 0 op/s 2024-03-24T11:33:21.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:21 smithi139 ceph-mon[91894]: pgmap v377: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 858 B/s rd, 0 op/s 2024-03-24T11:33:23.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:23 smithi062 ceph-mon[90553]: pgmap v378: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 858 B/s rd, 0 op/s 2024-03-24T11:33:23.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:23 smithi062 ceph-mon[98643]: pgmap v378: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 858 B/s rd, 0 op/s 2024-03-24T11:33:23.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:23 smithi139 ceph-mon[91894]: pgmap v378: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 858 B/s rd, 0 op/s 2024-03-24T11:33:25.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:25 smithi062 ceph-mon[90553]: pgmap v379: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T11:33:25.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:25 smithi062 ceph-mon[98643]: pgmap v379: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T11:33:25.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:25 smithi139 ceph-mon[91894]: pgmap v379: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T11:33:27.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:27.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:33:27.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:27 smithi062 ceph-mon[90553]: from='client.44892 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:33:27.406 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:27 smithi062 ceph-mon[90553]: pgmap v380: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 858 B/s rd, 0 op/s 2024-03-24T11:33:27.407 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:27.407 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:33:27.407 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:27 smithi062 ceph-mon[98643]: from='client.44892 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:33:27.407 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:27 smithi062 ceph-mon[98643]: pgmap v380: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 858 B/s rd, 0 op/s 2024-03-24T11:33:27.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:27.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:33:27.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:27 smithi139 ceph-mon[91894]: from='client.44892 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:33:27.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:27 smithi139 ceph-mon[91894]: pgmap v380: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 858 B/s rd, 0 op/s 2024-03-24T11:33:29.392 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:29 smithi062 ceph-mon[90553]: pgmap v381: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:33:29.393 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:29 smithi062 ceph-mon[98643]: pgmap v381: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:33:29.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:29 smithi139 ceph-mon[91894]: pgmap v381: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:33:30.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:30.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:30.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:30.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:30.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:30.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:30.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:30.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:30.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:31.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:33:30 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:33:30] "GET /metrics HTTP/1.1" 200 38839 "" "Prometheus/2.33.4" 2024-03-24T11:33:31.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:31 smithi139 ceph-mon[91894]: pgmap v382: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:33:31.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:31 smithi062 ceph-mon[98643]: pgmap v382: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:33:31.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:31 smithi062 ceph-mon[90553]: pgmap v382: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:33:33.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:33 smithi062 ceph-mon[98643]: pgmap v383: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:33:33.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:33 smithi062 ceph-mon[90553]: pgmap v383: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:33:33.533 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:33 smithi139 ceph-mon[91894]: pgmap v383: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:33:35.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:35.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:35.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:35.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:35 smithi139 ceph-mon[91894]: pgmap v384: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:33:35.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:35.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:35.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:35.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:35 smithi062 ceph-mon[90553]: pgmap v384: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:33:35.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:35.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:35.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:35.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:35 smithi062 ceph-mon[98643]: pgmap v384: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:33:37.411 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:37 smithi139 ceph-mon[91894]: from='client.44892 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:33:37.411 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:37 smithi139 ceph-mon[91894]: pgmap v385: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:33:37.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:37 smithi062 ceph-mon[98643]: from='client.44892 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:33:37.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:37 smithi062 ceph-mon[98643]: pgmap v385: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:33:37.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:37 smithi062 ceph-mon[90553]: from='client.44892 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:33:37.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:37 smithi062 ceph-mon[90553]: pgmap v385: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:33:39.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:39 smithi062 ceph-mon[90553]: pgmap v386: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:33:39.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:39 smithi062 ceph-mon[98643]: pgmap v386: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:33:39.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:39 smithi139 ceph-mon[91894]: pgmap v386: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:33:41.140 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:33:40 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:33:40] "GET /metrics HTTP/1.1" 200 38839 "" "Prometheus/2.33.4" 2024-03-24T11:33:41.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:41 smithi062 ceph-mon[90553]: pgmap v387: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:33:41.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:41 smithi062 ceph-mon[98643]: pgmap v387: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:33:41.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:41 smithi139 ceph-mon[91894]: pgmap v387: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:33:42.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:42 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:33:42.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:42 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:33:42.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:42 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:33:43.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:43 smithi062 ceph-mon[90553]: pgmap v388: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:33:43.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:43 smithi062 ceph-mon[98643]: pgmap v388: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:33:43.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:43 smithi139 ceph-mon[91894]: pgmap v388: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:33:45.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:45 smithi062 ceph-mon[90553]: pgmap v389: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:33:45.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:45 smithi062 ceph-mon[98643]: pgmap v389: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:33:45.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:45 smithi139 ceph-mon[91894]: pgmap v389: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:33:47.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:47 smithi139 ceph-mon[91894]: from='client.44892 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:33:47.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:47 smithi139 ceph-mon[91894]: pgmap v390: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:33:47.416 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:47 smithi062 ceph-mon[90553]: from='client.44892 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:33:47.416 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:47 smithi062 ceph-mon[90553]: pgmap v390: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:33:47.416 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:47 smithi062 ceph-mon[98643]: from='client.44892 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:33:47.416 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:47 smithi062 ceph-mon[98643]: pgmap v390: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:33:49.416 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:49 smithi062 ceph-mon[90553]: pgmap v391: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:33:49.417 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:49 smithi062 ceph-mon[98643]: pgmap v391: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:33:49.441 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:49 smithi139 ceph-mon[91894]: pgmap v391: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:33:50.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:50 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:50.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:50 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:50.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:50 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:50.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:50 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:50.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:50 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:50.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:50 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:50.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:50 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:50.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:50 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:50.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:50 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:51.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:33:50 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:33:50] "GET /metrics HTTP/1.1" 200 38847 "" "Prometheus/2.33.4" 2024-03-24T11:33:51.681 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:51 smithi139 ceph-mon[91894]: pgmap v392: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:33:51.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:51 smithi062 ceph-mon[90553]: pgmap v392: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:33:51.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:51 smithi062 ceph-mon[98643]: pgmap v392: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:33:53.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:53 smithi139 ceph-mon[91894]: pgmap v393: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:33:53.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:53 smithi062 ceph-mon[98643]: pgmap v393: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:33:53.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:53 smithi062 ceph-mon[90553]: pgmap v393: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:33:55.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:55.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:55.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:55 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:55.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:55 smithi062 ceph-mon[90553]: pgmap v394: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:33:55.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:55.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:55.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:55 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:55.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:55 smithi062 ceph-mon[98643]: pgmap v394: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:33:55.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:55.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:55.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:55 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:33:55.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:55 smithi139 ceph-mon[91894]: pgmap v394: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:33:56.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:56 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:33:56.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:56 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:33:56.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:56 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:33:57.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:57 smithi062 ceph-mon[90553]: from='client.44892 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:33:57.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:57 smithi062 ceph-mon[90553]: pgmap v395: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:33:57.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:57 smithi062 ceph-mon[98643]: from='client.44892 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:33:57.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:57 smithi062 ceph-mon[98643]: pgmap v395: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:33:57.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:57 smithi139 ceph-mon[91894]: from='client.44892 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:33:57.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:57 smithi139 ceph-mon[91894]: pgmap v395: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:33:59.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:33:59 smithi062 ceph-mon[98643]: pgmap v396: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:33:59.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:33:59 smithi062 ceph-mon[90553]: pgmap v396: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:33:59.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:33:59 smithi139 ceph-mon[91894]: pgmap v396: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:34:01.142 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:34:00 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:34:00] "GET /metrics HTTP/1.1" 200 38847 "" "Prometheus/2.33.4" 2024-03-24T11:34:01.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:01 smithi062 ceph-mon[90553]: pgmap v397: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:34:01.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:01 smithi062 ceph-mon[98643]: pgmap v397: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:34:01.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:01 smithi139 ceph-mon[91894]: pgmap v397: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:34:03.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:03 smithi062 ceph-mon[98643]: pgmap v398: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:34:03.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:03 smithi062 ceph-mon[90553]: pgmap v398: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:34:03.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:03 smithi139 ceph-mon[91894]: pgmap v398: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:34:04.416 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph orch ps' 2024-03-24T11:34:05.475 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:05 smithi062 ceph-mon[90553]: pgmap v399: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:34:05.476 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:05 smithi062 ceph-mon[98643]: pgmap v399: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:34:05.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:05 smithi139 ceph-mon[91894]: pgmap v399: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:34:06.651 INFO:teuthology.orchestra.run.smithi062.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-24T11:34:06.651 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi062 smithi062 running 12s ago 23m - - 2024-03-24T11:34:06.651 INFO:teuthology.orchestra.run.smithi062.stdout:agent.smithi139 smithi139 running 17s ago 23m - - 2024-03-24T11:34:06.651 INFO:teuthology.orchestra.run.smithi062.stdout:alertmanager.a smithi062 *:9093,9094 running (13m) 12s ago 18m - - 0.23.0 ba2b418f427c 818298d58140 2024-03-24T11:34:06.651 INFO:teuthology.orchestra.run.smithi062.stdout:grafana.a smithi139 *:3000 running (12m) 17s ago 17m 53.2M - 8.3.5 dad864ee21e9 b48167e7171e 2024-03-24T11:34:06.651 INFO:teuthology.orchestra.run.smithi062.stdout:iscsi.foo.smithi062.dggsvk smithi062 running (71s) 12s ago 16m - - 3.7 4f325f8440e9 270cf73ccabd 2024-03-24T11:34:06.651 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.x smithi139 *:8443,9283,8765 running (10m) 17s ago 22m 454M - 19.0.0-1667-gdb0330b1 4f325f8440e9 d94987b11387 2024-03-24T11:34:06.651 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.y smithi062 *:8443,9283,8765 running (12m) 12s ago 24m - - 19.0.0-1667-gdb0330b1 4f325f8440e9 ca1a49028d65 2024-03-24T11:34:06.651 INFO:teuthology.orchestra.run.smithi062.stdout:mon.a smithi062 running (10m) 12s ago 24m - 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 f7afe6e28c88 2024-03-24T11:34:06.651 INFO:teuthology.orchestra.run.smithi062.stdout:mon.b smithi139 running (9m) 17s ago 23m 58.7M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 ca357beea4dd 2024-03-24T11:34:06.651 INFO:teuthology.orchestra.run.smithi062.stdout:mon.c smithi062 running (9m) 12s ago 23m - 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 459e7449ec39 2024-03-24T11:34:06.651 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.a smithi062 *:9100 running (13m) 12s ago 18m - - 1.3.1 1dbe0e931976 79d5a02a999f 2024-03-24T11:34:06.652 INFO:teuthology.orchestra.run.smithi062.stdout:node-exporter.b smithi139 *:9100 running (12m) 17s ago 18m 15.6M - 1.3.1 1dbe0e931976 8f9f4608e465 2024-03-24T11:34:06.652 INFO:teuthology.orchestra.run.smithi062.stdout:osd.0 smithi062 running (8m) 12s ago 22m - 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 dfe4920b49c4 2024-03-24T11:34:06.652 INFO:teuthology.orchestra.run.smithi062.stdout:osd.1 smithi062 running (7m) 12s ago 22m - 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 c443b3fe97c1 2024-03-24T11:34:06.652 INFO:teuthology.orchestra.run.smithi062.stdout:osd.2 smithi062 running (6m) 12s ago 21m - 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 0eba40dc96b3 2024-03-24T11:34:06.652 INFO:teuthology.orchestra.run.smithi062.stdout:osd.3 smithi062 running (5m) 12s ago 21m - 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 7501d2e4c718 2024-03-24T11:34:06.652 INFO:teuthology.orchestra.run.smithi062.stdout:osd.4 smithi139 running (4m) 17s ago 20m 60.2M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 5d0f7f05ec6d 2024-03-24T11:34:06.652 INFO:teuthology.orchestra.run.smithi062.stdout:osd.5 smithi139 running (3m) 17s ago 20m 49.2M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 4ea95bfd2127 2024-03-24T11:34:06.652 INFO:teuthology.orchestra.run.smithi062.stdout:osd.6 smithi139 running (3m) 17s ago 20m 50.4M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 c67fd9426ac5 2024-03-24T11:34:06.652 INFO:teuthology.orchestra.run.smithi062.stdout:osd.7 smithi139 running (2m) 17s ago 19m 49.3M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 d4fbd9bccd09 2024-03-24T11:34:06.652 INFO:teuthology.orchestra.run.smithi062.stdout:prometheus.a smithi139 *:9095 running (11m) 17s ago 19m 71.3M - 2.33.4 514e6a882f6e 2ed47f810d33 2024-03-24T11:34:06.652 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi062.sacpds smithi062 *:8000 running (2m) 12s ago 17m - - 19.0.0-1667-gdb0330b1 4f325f8440e9 bdccb278a0c3 2024-03-24T11:34:06.652 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.foo.smithi139.yljesh smithi139 *:8000 running (119s) 17s ago 17m 76.2M - 19.0.0-1667-gdb0330b1 4f325f8440e9 9c325556e724 2024-03-24T11:34:06.652 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi062.xvpxnk smithi062 *:80 running (2m) 12s ago 17m - - 19.0.0-1667-gdb0330b1 4f325f8440e9 88bcf6d3dca0 2024-03-24T11:34:06.652 INFO:teuthology.orchestra.run.smithi062.stdout:rgw.smpl.smithi139.pbtrmp smithi139 *:80 running (110s) 17s ago 17m 77.9M - 19.0.0-1667-gdb0330b1 4f325f8440e9 c9bd2dd9ce30 2024-03-24T11:34:07.331 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:07 smithi062 ceph-mon[90553]: from='client.44946 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:34:07.331 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:07 smithi062 ceph-mon[90553]: from='client.44892 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:34:07.331 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:07 smithi062 ceph-mon[90553]: pgmap v400: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:34:07.332 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:07 smithi062 ceph-mon[98643]: from='client.44946 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:34:07.332 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:07 smithi062 ceph-mon[98643]: from='client.44892 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:34:07.332 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:07 smithi062 ceph-mon[98643]: pgmap v400: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:34:07.333 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph versions' 2024-03-24T11:34:07.410 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:07 smithi139 ceph-mon[91894]: from='client.44946 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:34:07.410 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:07 smithi139 ceph-mon[91894]: from='client.44892 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:34:07.410 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:07 smithi139 ceph-mon[91894]: pgmap v400: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:34:09.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:09 smithi062 ceph-mon[90553]: pgmap v401: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:34:09.466 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:09 smithi062 ceph-mon[98643]: pgmap v401: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:34:09.579 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:09 smithi139 ceph-mon[91894]: pgmap v401: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:34:10.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:34:10.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:34:10.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:10 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:34:10.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:34:10.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:34:10.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:10 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:34:10.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:34:10.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:34:10.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:10 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:34:10.878 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:34:10.878 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": { 2024-03-24T11:34:10.878 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 3 2024-03-24T11:34:10.878 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:34:10.878 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": { 2024-03-24T11:34:10.878 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-24T11:34:10.878 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:34:10.879 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": { 2024-03-24T11:34:10.879 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 8 2024-03-24T11:34:10.879 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:34:10.879 INFO:teuthology.orchestra.run.smithi062.stdout: "rgw": { 2024-03-24T11:34:10.879 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 4 2024-03-24T11:34:10.879 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-03-24T11:34:10.879 INFO:teuthology.orchestra.run.smithi062.stdout: "overall": { 2024-03-24T11:34:10.879 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 17 2024-03-24T11:34:10.879 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-03-24T11:34:10.879 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:34:11.227 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:34:10 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:34:10] "GET /metrics HTTP/1.1" 200 38847 "" "Prometheus/2.33.4" 2024-03-24T11:34:11.568 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph orch upgrade status' 2024-03-24T11:34:11.856 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:11 smithi062 ceph-mon[90553]: from='client.? 172.21.15.62:0/2135985929' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:34:11.856 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:11 smithi062 ceph-mon[90553]: pgmap v402: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:34:11.856 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:11 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:34:11.857 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:11 smithi062 ceph-mon[98643]: from='client.? 172.21.15.62:0/2135985929' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:34:11.857 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:11 smithi062 ceph-mon[98643]: pgmap v402: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:34:11.857 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:11 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:34:11.857 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:11 smithi139 ceph-mon[91894]: from='client.? 172.21.15.62:0/2135985929' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:34:11.857 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:11 smithi139 ceph-mon[91894]: pgmap v402: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:34:11.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:11 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:34:13.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:13 smithi062 ceph-mon[90553]: pgmap v403: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:34:13.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:13 smithi062 ceph-mon[98643]: pgmap v403: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:34:13.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:13 smithi139 ceph-mon[91894]: pgmap v403: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:34:14.114 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:34:14.114 INFO:teuthology.orchestra.run.smithi062.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69", 2024-03-24T11:34:14.114 INFO:teuthology.orchestra.run.smithi062.stdout: "in_progress": true, 2024-03-24T11:34:14.114 INFO:teuthology.orchestra.run.smithi062.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-24T11:34:14.114 INFO:teuthology.orchestra.run.smithi062.stdout: "services_complete": [ 2024-03-24T11:34:14.115 INFO:teuthology.orchestra.run.smithi062.stdout: "osd", 2024-03-24T11:34:14.115 INFO:teuthology.orchestra.run.smithi062.stdout: "mon", 2024-03-24T11:34:14.115 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr", 2024-03-24T11:34:14.115 INFO:teuthology.orchestra.run.smithi062.stdout: "iscsi", 2024-03-24T11:34:14.115 INFO:teuthology.orchestra.run.smithi062.stdout: "rgw" 2024-03-24T11:34:14.115 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-03-24T11:34:14.115 INFO:teuthology.orchestra.run.smithi062.stdout: "progress": "18/23 daemons upgraded", 2024-03-24T11:34:14.115 INFO:teuthology.orchestra.run.smithi062.stdout: "message": "Error: UPGRADE_REDEPLOY_DAEMON: Upgrading daemon iscsi.foo.smithi062.dggsvk on host smithi062 failed.", 2024-03-24T11:34:14.115 INFO:teuthology.orchestra.run.smithi062.stdout: "is_paused": true 2024-03-24T11:34:14.115 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:34:14.820 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph health detail' 2024-03-24T11:34:15.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:15 smithi062 ceph-mon[90553]: from='client.64927 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:34:15.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:34:15.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:34:15.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:15 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:34:15.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:15 smithi062 ceph-mon[90553]: pgmap v404: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:34:15.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:15 smithi062 ceph-mon[98643]: from='client.64927 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:34:15.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:15 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:34:15.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:15 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:34:15.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:15 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:34:15.728 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:15 smithi062 ceph-mon[98643]: pgmap v404: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:34:15.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:15 smithi139 ceph-mon[91894]: from='client.64927 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:34:15.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:15 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:34:15.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:15 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:34:15.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:15 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:34:15.859 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:15 smithi139 ceph-mon[91894]: pgmap v404: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:34:16.877 INFO:teuthology.orchestra.run.smithi062.stdout:HEALTH_WARN Upgrading daemon iscsi.foo.smithi062.dggsvk on host smithi062 failed. 2024-03-24T11:34:16.877 INFO:teuthology.orchestra.run.smithi062.stdout:[WRN] UPGRADE_REDEPLOY_DAEMON: Upgrading daemon iscsi.foo.smithi062.dggsvk on host smithi062 failed. 2024-03-24T11:34:16.877 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrade daemon: iscsi.foo.smithi062.dggsvk: cephadm exited with an error code: 1, stderr: Redeploy daemon iscsi.foo.smithi062.dggsvk ... 2024-03-24T11:34:16.877 INFO:teuthology.orchestra.run.smithi062.stdout:Creating ceph-iscsi config... 2024-03-24T11:34:16.877 INFO:teuthology.orchestra.run.smithi062.stdout:Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/iscsi-gateway.cfg 2024-03-24T11:34:16.877 INFO:teuthology.orchestra.run.smithi062.stdout:Write file: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/iscsi.foo.smithi062.dggsvk/tcmu-runner-entrypoint.sh 2024-03-24T11:34:16.877 INFO:teuthology.orchestra.run.smithi062.stdout:Failed to trim old cgroups /sys/fs/cgroup/system.slice/system-ceph\x2debb89568\x2de9ce\x2d11ee\x2d95cd\x2d87774f69a715.slice/ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service 2024-03-24T11:34:16.877 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 1 from systemctl start ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk 2024-03-24T11:34:16.877 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:34:16.877 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:34:16.877 INFO:teuthology.orchestra.run.smithi062.stdout:Traceback (most recent call last): 2024-03-24T11:34:16.877 INFO:teuthology.orchestra.run.smithi062.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-24T11:34:16.878 INFO:teuthology.orchestra.run.smithi062.stdout: return _run_code(code, main_globals, None, 2024-03-24T11:34:16.878 INFO:teuthology.orchestra.run.smithi062.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-24T11:34:16.878 INFO:teuthology.orchestra.run.smithi062.stdout: exec(code, run_globals) 2024-03-24T11:34:16.878 INFO:teuthology.orchestra.run.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5520, in 2024-03-24T11:34:16.878 INFO:teuthology.orchestra.run.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 5508, in main 2024-03-24T11:34:16.878 INFO:teuthology.orchestra.run.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3038, in command_deploy_from 2024-03-24T11:34:16.878 INFO:teuthology.orchestra.run.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3073, in _common_deploy 2024-03-24T11:34:16.878 INFO:teuthology.orchestra.run.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 3093, in _deploy_daemon_container 2024-03-24T11:34:16.878 INFO:teuthology.orchestra.run.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 1029, in deploy_daemon 2024-03-24T11:34:16.878 INFO:teuthology.orchestra.run.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/__main__.py", line 1165, in deploy_daemon_units 2024-03-24T11:34:16.878 INFO:teuthology.orchestra.run.smithi062.stdout: File "/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/cephadm.1da7fdc466b4c6904edf167cf65a395d23162ea50272e1f853080bb41ee7cd4a/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-03-24T11:34:16.878 INFO:teuthology.orchestra.run.smithi062.stdout:RuntimeError: Failed command: systemctl start ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk: Job for ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service failed because the control process exited with error code. 2024-03-24T11:34:16.878 INFO:teuthology.orchestra.run.smithi062.stdout:See "systemctl status ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" and "journalctl -xeu ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@iscsi.foo.smithi062.dggsvk.service" for details. 2024-03-24T11:34:17.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:16 smithi062 ceph-mon[90553]: from='client.? 172.21.15.62:0/4137049298' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:34:17.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:16 smithi062 ceph-mon[98643]: from='client.? 172.21.15.62:0/4137049298' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:34:17.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:16 smithi139 ceph-mon[91894]: from='client.? 172.21.15.62:0/4137049298' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-24T11:34:17.470 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-03-24T11:34:18.030 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:17 smithi062 ceph-mon[90553]: from='client.44892 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:34:18.030 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:17 smithi062 ceph-mon[90553]: pgmap v405: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:34:18.030 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:34:18.031 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:34:18.031 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:34:18.031 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:34:18.031 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:34:18.031 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:34:18.031 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:34:18.031 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:34:18.031 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:34:18.031 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:34:18.031 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:34:18.031 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:34:18.031 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:17 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:34:18.031 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:17 smithi062 ceph-mon[98643]: from='client.44892 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:34:18.032 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:17 smithi062 ceph-mon[98643]: pgmap v405: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:34:18.032 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:34:18.032 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:34:18.032 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:34:18.032 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:34:18.032 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:34:18.032 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:34:18.032 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:34:18.032 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:34:18.032 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:34:18.032 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:34:18.032 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:34:18.032 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:34:18.032 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:17 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:34:18.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:17 smithi139 ceph-mon[91894]: from='client.44892 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:34:18.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:17 smithi139 ceph-mon[91894]: pgmap v405: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:34:18.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:34:18.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:34:18.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:34:18.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:34:18.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:34:18.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:34:18.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:34:18.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T11:34:18.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:34:18.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:34:18.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T11:34:18.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T11:34:18.359 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:17 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:34:19.119 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:18 smithi062 ceph-mon[98643]: Adjusting osd_memory_target on smithi139 to 2736M 2024-03-24T11:34:19.120 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:18 smithi062 ceph-mon[98643]: Adjusting osd_memory_target on smithi062 to 2480M 2024-03-24T11:34:19.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:18 smithi062 ceph-mon[90553]: Adjusting osd_memory_target on smithi139 to 2736M 2024-03-24T11:34:19.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:18 smithi062 ceph-mon[90553]: Adjusting osd_memory_target on smithi062 to 2480M 2024-03-24T11:34:19.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:18 smithi139 ceph-mon[91894]: Adjusting osd_memory_target on smithi139 to 2736M 2024-03-24T11:34:19.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:18 smithi139 ceph-mon[91894]: Adjusting osd_memory_target on smithi062 to 2480M 2024-03-24T11:34:19.685 INFO:teuthology.orchestra.run.smithi062.stdout:true 2024-03-24T11:34:20.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:19 smithi062 ceph-mon[90553]: pgmap v406: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:34:20.227 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:19 smithi062 ceph-mon[90553]: from='client.? 172.21.15.62:0/3068115649' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:34:20.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:19 smithi062 ceph-mon[98643]: pgmap v406: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:34:20.227 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:19 smithi062 ceph-mon[98643]: from='client.? 172.21.15.62:0/3068115649' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:34:20.321 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-03-24T11:34:20.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:19 smithi139 ceph-mon[91894]: pgmap v406: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:34:20.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:19 smithi139 ceph-mon[91894]: from='client.? 172.21.15.62:0/3068115649' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:34:20.892 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:34:20 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:34:20] "GET /metrics HTTP/1.1" 200 38843 "" "Prometheus/2.33.4" 2024-03-24T11:34:21.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:21 smithi062 ceph-mon[90553]: pgmap v407: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:34:21.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:21 smithi062 ceph-mon[98643]: pgmap v407: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:34:21.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:21 smithi139 ceph-mon[91894]: pgmap v407: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:34:22.445 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)" 2024-03-24T11:34:22.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:22 smithi062 ceph-mon[90553]: from='client.? 172.21.15.62:0/3802131591' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:34:22.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:22 smithi062 ceph-mon[98643]: from='client.? 172.21.15.62:0/3802131591' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:34:22.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:22 smithi139 ceph-mon[91894]: from='client.? 172.21.15.62:0/3802131591' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-24T11:34:23.148 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph orch ls | grep '"'"'^osd '"'"'' 2024-03-24T11:34:23.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:23 smithi062 ceph-mon[98643]: pgmap v408: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:34:23.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:23 smithi062 ceph-mon[90553]: pgmap v408: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:34:23.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:23 smithi139 ceph-mon[91894]: pgmap v408: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:34:25.285 INFO:teuthology.orchestra.run.smithi062.stdout:osd 8 15s ago - 2024-03-24T11:34:25.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:25 smithi062 ceph-mon[90553]: pgmap v409: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:34:25.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:25 smithi062 ceph-mon[98643]: pgmap v409: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:34:25.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:25 smithi139 ceph-mon[91894]: pgmap v409: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:34:25.850 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-24T11:34:25.860 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi062.front.sepia.ceph.com 2024-03-24T11:34:25.860 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- bash -c 'ceph orch upgrade ls' 2024-03-24T11:34:26.431 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:26 smithi062 ceph-mon[98643]: from='client.64951 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:34:26.432 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:26 smithi062 ceph-mon[90553]: from='client.64951 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:34:26.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:26 smithi139 ceph-mon[91894]: from='client.64951 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:34:27.449 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:27 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:34:27.449 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:27 smithi062 ceph-mon[98643]: from='client.44892 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:34:27.449 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:27 smithi062 ceph-mon[98643]: pgmap v410: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:34:27.449 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:27 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:34:27.450 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:27 smithi062 ceph-mon[90553]: from='client.44892 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:34:27.450 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:27 smithi062 ceph-mon[90553]: pgmap v410: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:34:27.482 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:27 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T11:34:27.482 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:27 smithi139 ceph-mon[91894]: from='client.44892 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:34:27.482 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:27 smithi139 ceph-mon[91894]: pgmap v410: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:34:28.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:28 smithi062 ceph-mon[90553]: from='client.44988 -' entity='client.admin' cmd=[{"prefix": "orch upgrade ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:34:28.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:28 smithi062 ceph-mon[98643]: from='client.44988 -' entity='client.admin' cmd=[{"prefix": "orch upgrade ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:34:28.588 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-03-24T11:34:28.588 INFO:teuthology.orchestra.run.smithi062.stdout: "image": "quay.io/ceph/ceph", 2024-03-24T11:34:28.588 INFO:teuthology.orchestra.run.smithi062.stdout: "registry": "quay.io", 2024-03-24T11:34:28.588 INFO:teuthology.orchestra.run.smithi062.stdout: "bare_image": "ceph/ceph", 2024-03-24T11:34:28.588 INFO:teuthology.orchestra.run.smithi062.stdout: "versions": [] 2024-03-24T11:34:28.588 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-03-24T11:34:28.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:28 smithi139 ceph-mon[91894]: from='client.44988 -' entity='client.admin' cmd=[{"prefix": "orch upgrade ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:34:29.145 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- bash -c 'ceph orch upgrade ls --image quay.io/ceph/ceph --show-all-versions | grep 16.2.0' 2024-03-24T11:34:29.444 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:29 smithi062 ceph-mon[90553]: pgmap v411: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:34:29.444 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:29 smithi062 ceph-mon[98643]: pgmap v411: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:34:29.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:29 smithi139 ceph-mon[91894]: pgmap v411: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:34:30.805 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:34:30.805 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:34:30.805 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:30 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:34:30.805 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:34:30.805 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:34:30.805 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:30 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:34:31.105 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:34:30 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:34:30] "GET /metrics HTTP/1.1" 200 38843 "" "Prometheus/2.33.4" 2024-03-24T11:34:31.107 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:34:31.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:34:31.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:30 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:34:31.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:31 smithi062 ceph-mon[90553]: pgmap v412: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:34:31.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:31 smithi062 ceph-mon[98643]: pgmap v412: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:34:32.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:31 smithi139 ceph-mon[91894]: pgmap v412: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:34:33.324 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:33 smithi062 ceph-mon[90553]: from='client.44994 -' entity='client.admin' cmd=[{"prefix": "orch upgrade ls", "image": "quay.io/ceph/ceph", "show_all_versions": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:34:33.324 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:33 smithi062 ceph-mon[90553]: pgmap v413: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:34:33.324 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:33 smithi062 ceph-mon[98643]: from='client.44994 -' entity='client.admin' cmd=[{"prefix": "orch upgrade ls", "image": "quay.io/ceph/ceph", "show_all_versions": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:34:33.324 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:33 smithi062 ceph-mon[98643]: pgmap v413: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:34:33.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:33 smithi139 ceph-mon[91894]: from='client.44994 -' entity='client.admin' cmd=[{"prefix": "orch upgrade ls", "image": "quay.io/ceph/ceph", "show_all_versions": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:34:33.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:33 smithi139 ceph-mon[91894]: pgmap v413: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:34:33.821 INFO:teuthology.orchestra.run.smithi062.stdout: "16.2.0", 2024-03-24T11:34:34.360 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 -- bash -c 'ceph orch upgrade ls --image quay.io/ceph/ceph --tags | grep v16.2.2' 2024-03-24T11:34:35.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:34:35.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:34:35.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:35 smithi062 ceph-mon[90553]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:34:35.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:35 smithi062 ceph-mon[90553]: pgmap v414: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:34:35.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:34:35.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:34:35.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:35 smithi062 ceph-mon[98643]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:34:35.977 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:35 smithi062 ceph-mon[98643]: pgmap v414: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:34:36.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:34:36.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:34:36.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:35 smithi139 ceph-mon[91894]: from='mgr.25489 172.21.15.62:0/1248867014' entity='mgr.y' 2024-03-24T11:34:36.108 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:35 smithi139 ceph-mon[91894]: pgmap v414: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T11:34:37.048 INFO:teuthology.orchestra.run.smithi062.stdout: "v16.2.2", 2024-03-24T11:34:37.048 INFO:teuthology.orchestra.run.smithi062.stdout: "v16.2.2-20210505", 2024-03-24T11:34:37.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:37 smithi062 ceph-mon[90553]: from='client.45000 -' entity='client.admin' cmd=[{"prefix": "orch upgrade ls", "image": "quay.io/ceph/ceph", "tags": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:34:37.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:37 smithi062 ceph-mon[90553]: from='client.44892 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:34:37.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:37 smithi062 ceph-mon[90553]: pgmap v415: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:34:37.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:37 smithi062 ceph-mon[98643]: from='client.45000 -' entity='client.admin' cmd=[{"prefix": "orch upgrade ls", "image": "quay.io/ceph/ceph", "tags": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:34:37.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:37 smithi062 ceph-mon[98643]: from='client.44892 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:34:37.477 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:37 smithi062 ceph-mon[98643]: pgmap v415: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:34:37.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:37 smithi139 ceph-mon[91894]: from='client.45000 -' entity='client.admin' cmd=[{"prefix": "orch upgrade ls", "image": "quay.io/ceph/ceph", "tags": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T11:34:37.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:37 smithi139 ceph-mon[91894]: from='client.44892 -' entity='client.iscsi.foo.smithi062.dggsvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T11:34:37.608 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:37 smithi139 ceph-mon[91894]: pgmap v415: 161 pgs: 161 active+clean; 585 KiB data, 282 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T11:34:37.623 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-24T11:34:37.632 INFO:tasks.cephadm:Teardown begin 2024-03-24T11:34:37.633 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-24T11:34:37.663 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-24T11:34:37.701 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-24T11:34:37.701 DEBUG:teuthology.orchestra.run.smithi062:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-24T11:34:37.718 DEBUG:teuthology.orchestra.run.smithi139:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-24T11:34:37.757 INFO:tasks.cephadm:Stopping all daemons... 2024-03-24T11:34:37.757 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-03-24T11:34:37.757 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mon.a 2024-03-24T11:34:37.977 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:37 smithi062 systemd[1]: Stopping Ceph mon.a for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:34:38.327 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:38 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-a[90549]: 2024-03-24T11:34:38.061+0000 7f2d82f5b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T11:34:38.327 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:38 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-a[90549]: 2024-03-24T11:34:38.061+0000 7f2d82f5b700 -1 mon.a@0(leader) e4 *** Got Signal Terminated *** 2024-03-24T11:34:38.327 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:34:38 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: [24/Mar/2024:11:34:38] ENGINE Bus STOPPING 2024-03-24T11:34:38.327 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:34:38 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: [24/Mar/2024:11:34:38] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-03-24T11:34:38.327 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:34:38 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: [24/Mar/2024:11:34:38] ENGINE Bus STOPPED 2024-03-24T11:34:38.327 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:34:38 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: [24/Mar/2024:11:34:38] ENGINE Bus STARTING 2024-03-24T11:34:38.327 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:34:38 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: CherryPy Checker: 2024-03-24T11:34:38.327 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:34:38 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: The Application mounted at '' has an empty config. 2024-03-24T11:34:38.327 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:34:38 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: 2024-03-24T11:34:38.328 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:34:38 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: [24/Mar/2024:11:34:38] ENGINE Serving on http://:::9283 2024-03-24T11:34:38.328 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:34:38 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: [24/Mar/2024:11:34:38] ENGINE Bus STARTED 2024-03-24T11:34:38.629 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:38 smithi062 podman[179722]: 2024-03-24 11:34:38.327049728 +0000 UTC m=+0.379915663 container died f7afe6e28c88760a16d29c35f0ce7746b2d2e57a5739780da2869d121bf30c0f (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-a, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-db0330b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-24T11:34:38.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:38 smithi062 podman[179722]: 2024-03-24 11:34:38.728435122 +0000 UTC m=+0.781301065 container cleanup f7afe6e28c88760a16d29c35f0ce7746b2d2e57a5739780da2869d121bf30c0f (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-a, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T11:34:38.965 INFO:journalctl@ceph.mon.a.smithi062.stdout:Mar 24 11:34:38 smithi062 bash[179722]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-a 2024-03-24T11:34:39.079 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mon.a.service' 2024-03-24T11:34:39.130 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T11:34:39.130 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-03-24T11:34:39.130 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-03-24T11:34:39.131 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mon.c 2024-03-24T11:34:39.226 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:39 smithi062 systemd[1]: Stopping Ceph mon.c for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:34:39.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:39 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-c[98639]: 2024-03-24T11:34:39.415+0000 7ffbac9d2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T11:34:39.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:39 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-c[98639]: 2024-03-24T11:34:39.415+0000 7ffbac9d2700 -1 mon.c@1(peon) e4 *** Got Signal Terminated *** 2024-03-24T11:34:39.727 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:39 smithi062 podman[179856]: 2024-03-24 11:34:39.528133914 +0000 UTC m=+0.209982912 container died 459e7449ec3927a943e5a6d4eb9682d730abe01433b21ef9dc53de0ce9027d0d (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-c, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-24T11:34:39.727 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:34:39 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: [24/Mar/2024:11:34:39] ENGINE Bus STOPPING 2024-03-24T11:34:39.727 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:34:39 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: [24/Mar/2024:11:34:39] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-03-24T11:34:39.728 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:34:39 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: [24/Mar/2024:11:34:39] ENGINE Bus STOPPED 2024-03-24T11:34:39.728 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:34:39 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: [24/Mar/2024:11:34:39] ENGINE Bus STARTING 2024-03-24T11:34:39.728 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:34:39 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: [24/Mar/2024:11:34:39] ENGINE Serving on http://:::9283 2024-03-24T11:34:39.728 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:34:39 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: [24/Mar/2024:11:34:39] ENGINE Bus STARTED 2024-03-24T11:34:40.067 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:39 smithi062 podman[179856]: 2024-03-24 11:34:39.857494346 +0000 UTC m=+0.539343343 container cleanup 459e7449ec3927a943e5a6d4eb9682d730abe01433b21ef9dc53de0ce9027d0d (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-c, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-24T11:34:40.067 INFO:journalctl@ceph.mon.c.smithi062.stdout:Mar 24 11:34:39 smithi062 bash[179856]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-c 2024-03-24T11:34:40.201 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mon.c.service' 2024-03-24T11:34:40.237 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T11:34:40.237 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-03-24T11:34:40.237 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-03-24T11:34:40.237 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mon.b 2024-03-24T11:34:40.559 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:40 smithi139 systemd[1]: Stopping Ceph mon.b for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:34:40.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:40 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-b[91890]: 2024-03-24T11:34:40.558+0000 7fee0704b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T11:34:40.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:40 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-b[91890]: 2024-03-24T11:34:40.558+0000 7fee0704b700 -1 mon.b@2(peon) e4 *** Got Signal Terminated *** 2024-03-24T11:34:40.858 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:40 smithi139 podman[154586]: 2024-03-24 11:34:40.661248415 +0000 UTC m=+0.224952628 container died ca357beea4dd13c6c6cc4a856ed836fe6e96b3e966014469650898c4b3682bfe (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240307) 2024-03-24T11:34:41.226 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:34:40 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y[69438]: ::ffff:172.21.15.139 - - [24/Mar/2024:11:34:40] "GET /metrics HTTP/1.1" 200 38843 "" "Prometheus/2.33.4" 2024-03-24T11:34:41.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:41 smithi139 podman[154586]: 2024-03-24 11:34:41.124983753 +0000 UTC m=+0.688687980 container cleanup ca357beea4dd13c6c6cc4a856ed836fe6e96b3e966014469650898c4b3682bfe (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-b, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-24T11:34:41.358 INFO:journalctl@ceph.mon.b.smithi139.stdout:Mar 24 11:34:41 smithi139 bash[154586]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mon-b 2024-03-24T11:34:41.547 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mon.b.service' 2024-03-24T11:34:41.591 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T11:34:41.592 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-03-24T11:34:41.592 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-03-24T11:34:41.592 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mgr.y 2024-03-24T11:34:41.919 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:34:41 smithi062 systemd[1]: Stopping Ceph mgr.y for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:34:42.214 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:34:41 smithi062 podman[179991]: 2024-03-24 11:34:41.928731998 +0000 UTC m=+0.145039628 container died ca1a49028d652cca6e2c7762d3a815390dfe41049205724bca556a6566cec60b (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-db0330b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, io.buildah.version=1.33.5) 2024-03-24T11:34:42.476 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:34:42 smithi062 podman[179991]: 2024-03-24 11:34:42.429762286 +0000 UTC m=+0.646069916 container cleanup ca1a49028d652cca6e2c7762d3a815390dfe41049205724bca556a6566cec60b (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y, RELEASE=squid-db0330b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-24T11:34:42.477 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:34:42 smithi062 bash[179991]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y 2024-03-24T11:34:43.114 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mgr.y.service' 2024-03-24T11:34:43.137 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:34:42 smithi062 podman[180006]: 2024-03-24 11:34:42.864795028 +0000 UTC m=+0.940770631 container remove ca1a49028d652cca6e2c7762d3a815390dfe41049205724bca556a6566cec60b (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-y, GIT_CLEAN=True, org.label-schema.build-date=20240307, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-24T11:34:43.137 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:34:43 smithi062 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mgr.y.service: Deactivated successfully. 2024-03-24T11:34:43.138 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:34:43 smithi062 systemd[1]: Stopped Ceph mgr.y for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:34:43.138 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Mar 24 11:34:43 smithi062 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mgr.y.service: Consumed 5min 7.774s CPU time. 2024-03-24T11:34:43.151 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T11:34:43.151 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-03-24T11:34:43.151 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-03-24T11:34:43.151 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mgr.x 2024-03-24T11:34:43.468 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:34:43 smithi139 systemd[1]: Stopping Ceph mgr.x for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:34:43.821 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:34:43 smithi139 podman[154707]: 2024-03-24 11:34:43.468216577 +0000 UTC m=+0.124328071 container died d94987b11387352d4adcf75c06f5c3a66cc0dce69cbe5db9b20f3f0acf862650 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=squid-db0330b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240307) 2024-03-24T11:34:43.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:34:43 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[64072]: level=warn ts=2024-03-24T11:34:43.554Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi062.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.62:8443: connect: connection refused" 2024-03-24T11:34:43.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:34:43 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[64072]: level=warn ts=2024-03-24T11:34:43.554Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi139.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.139:8443: connect: connection refused" 2024-03-24T11:34:44.108 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:34:43 smithi139 podman[154707]: 2024-03-24 11:34:43.82061735 +0000 UTC m=+0.476728908 container cleanup d94987b11387352d4adcf75c06f5c3a66cc0dce69cbe5db9b20f3f0acf862650 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-db0330b, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T11:34:44.108 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:34:43 smithi139 bash[154707]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x 2024-03-24T11:34:44.108 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:34:44 smithi139 podman[154721]: 2024-03-24 11:34:44.022437719 +0000 UTC m=+0.553148791 container remove d94987b11387352d4adcf75c06f5c3a66cc0dce69cbe5db9b20f3f0acf862650 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-mgr-x, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-db0330b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T11:34:44.108 INFO:journalctl@ceph.mgr.x.smithi139.stdout:Mar 24 11:34:44 smithi139 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mgr.x.service: Main process exited, code=exited, status=143/n/a 2024-03-24T11:34:44.245 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@mgr.x.service' 2024-03-24T11:34:44.282 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T11:34:44.282 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-03-24T11:34:44.283 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-24T11:34:44.283 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.0 2024-03-24T11:34:44.600 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:34:44 smithi062 systemd[1]: Stopping Ceph osd.0 for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:34:44.977 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:34:44 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0[113580]: 2024-03-24T11:34:44.599+0000 7f0a59db7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T11:34:44.977 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:34:44 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0[113580]: 2024-03-24T11:34:44.599+0000 7f0a59db7700 -1 osd.0 180 *** Got signal Terminated *** 2024-03-24T11:34:44.977 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:34:44 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0[113580]: 2024-03-24T11:34:44.600+0000 7f0a59db7700 -1 osd.0 180 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T11:34:50.184 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:34:49 smithi062 podman[180115]: 2024-03-24 11:34:49.91549219 +0000 UTC m=+5.442838017 container died dfe4920b49c4d7a791c09566e939cd2e31f60c9788ae7ac0f7c0bc049d2fb1a1 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, RELEASE=squid-db0330b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-24T11:34:50.444 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:34:50 smithi062 podman[180115]: 2024-03-24 11:34:50.184185836 +0000 UTC m=+5.711531642 container cleanup dfe4920b49c4d7a791c09566e939cd2e31f60c9788ae7ac0f7c0bc049d2fb1a1 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True) 2024-03-24T11:34:50.444 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:34:50 smithi062 bash[180115]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0 2024-03-24T11:34:50.726 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:34:50 smithi062 podman[180132]: 2024-03-24 11:34:50.444036774 +0000 UTC m=+0.529499851 container remove dfe4920b49c4d7a791c09566e939cd2e31f60c9788ae7ac0f7c0bc049d2fb1a1 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240307, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-03-24T11:34:52.221 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:34:52 smithi062 podman[180289]: 2024-03-24 11:34:52.098045921 +0000 UTC m=+0.020498429 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:34:52.221 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:34:52 smithi062 podman[180289]: 2024-03-24 11:34:52.220891678 +0000 UTC m=+0.143344194 container create 6c78fd0916c36ca5159e0c46a6eca572261c262ff9bf41ccdea42a953efc04e1 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-24T11:34:52.727 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:34:52 smithi062 podman[180289]: 2024-03-24 11:34:52.473157323 +0000 UTC m=+0.395609838 container init 6c78fd0916c36ca5159e0c46a6eca572261c262ff9bf41ccdea42a953efc04e1 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-db0330b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS) 2024-03-24T11:34:52.727 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:34:52 smithi062 podman[180289]: 2024-03-24 11:34:52.477016461 +0000 UTC m=+0.399468977 container start 6c78fd0916c36ca5159e0c46a6eca572261c262ff9bf41ccdea42a953efc04e1 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux ) 2024-03-24T11:34:52.727 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:34:52 smithi062 podman[180289]: 2024-03-24 11:34:52.539990838 +0000 UTC m=+0.462443352 container attach 6c78fd0916c36ca5159e0c46a6eca572261c262ff9bf41ccdea42a953efc04e1 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-db0330b, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T11:34:53.086 INFO:journalctl@ceph.osd.0.smithi062.stdout:Mar 24 11:34:52 smithi062 podman[180289]: 2024-03-24 11:34:52.804670283 +0000 UTC m=+0.727122793 container died 6c78fd0916c36ca5159e0c46a6eca572261c262ff9bf41ccdea42a953efc04e1 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-0-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-db0330b, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T11:34:53.382 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.0.service' 2024-03-24T11:34:53.423 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T11:34:53.423 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-24T11:34:53.423 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-24T11:34:53.423 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.1 2024-03-24T11:34:53.713 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:34:53 smithi062 systemd[1]: Stopping Ceph osd.1 for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:34:53.714 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:34:53 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[64072]: level=error ts=2024-03-24T11:34:53.553Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=6 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi062.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.62:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi139.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.139:8443: connect: connection refused" 2024-03-24T11:34:53.714 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:34:53 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[64072]: level=warn ts=2024-03-24T11:34:53.554Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi062.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.62:8443: connect: connection refused" 2024-03-24T11:34:53.714 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:34:53 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[64072]: level=warn ts=2024-03-24T11:34:53.555Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi139.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.139:8443: connect: connection refused" 2024-03-24T11:34:54.477 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:34:54 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1[124909]: 2024-03-24T11:34:54.174+0000 7f5f104e3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T11:34:54.477 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:34:54 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1[124909]: 2024-03-24T11:34:54.174+0000 7f5f104e3700 -1 osd.1 180 *** Got signal Terminated *** 2024-03-24T11:34:54.477 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:34:54 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1[124909]: 2024-03-24T11:34:54.174+0000 7f5f104e3700 -1 osd.1 180 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T11:34:59.871 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:34:59 smithi062 podman[180553]: 2024-03-24 11:34:59.579370371 +0000 UTC m=+5.498371940 container died c443b3fe97c1affcb852f369d64fd050cfad2baa18a8dfbc554a5b3785d9949a (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-24T11:35:00.227 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:34:59 smithi062 podman[180553]: 2024-03-24 11:34:59.87090324 +0000 UTC m=+5.789904810 container cleanup c443b3fe97c1affcb852f369d64fd050cfad2baa18a8dfbc554a5b3785d9949a (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-24T11:35:00.227 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:34:59 smithi062 bash[180553]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1 2024-03-24T11:35:00.227 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:35:00 smithi062 podman[181130]: 2024-03-24 11:35:00.097000635 +0000 UTC m=+0.511257276 container remove c443b3fe97c1affcb852f369d64fd050cfad2baa18a8dfbc554a5b3785d9949a (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-db0330b, org.label-schema.build-date=20240307) 2024-03-24T11:35:00.692 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:35:00 smithi062 podman[181221]: 2024-03-24 11:35:00.334709384 +0000 UTC m=+0.017589432 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:35:00.692 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:35:00 smithi062 podman[181221]: 2024-03-24 11:35:00.472845509 +0000 UTC m=+0.155725579 container create 5d32c634e271117c289f7539d6d43f59f4cdbf0dc5a35d9444ed7de7257cbb60 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240307) 2024-03-24T11:35:00.692 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:35:00 smithi062 podman[181221]: 2024-03-24 11:35:00.692586841 +0000 UTC m=+0.375466877 container init 5d32c634e271117c289f7539d6d43f59f4cdbf0dc5a35d9444ed7de7257cbb60 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=squid-db0330b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-24T11:35:00.977 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:35:00 smithi062 podman[181221]: 2024-03-24 11:35:00.696137937 +0000 UTC m=+0.379017974 container start 5d32c634e271117c289f7539d6d43f59f4cdbf0dc5a35d9444ed7de7257cbb60 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-db0330b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-24T11:35:00.977 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:35:00 smithi062 podman[181221]: 2024-03-24 11:35:00.783739008 +0000 UTC m=+0.466619036 container attach 5d32c634e271117c289f7539d6d43f59f4cdbf0dc5a35d9444ed7de7257cbb60 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, RELEASE=squid-db0330b) 2024-03-24T11:35:01.444 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:35:00 smithi062 podman[181221]: 2024-03-24 11:35:00.998311818 +0000 UTC m=+0.681191858 container died 5d32c634e271117c289f7539d6d43f59f4cdbf0dc5a35d9444ed7de7257cbb60 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-db0330b, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-24T11:35:01.693 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.1.service' 2024-03-24T11:35:01.720 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:35:01 smithi062 podman[181248]: 2024-03-24 11:35:01.678599408 +0000 UTC m=+0.675700517 container remove 5d32c634e271117c289f7539d6d43f59f4cdbf0dc5a35d9444ed7de7257cbb60 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-1-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, GIT_CLEAN=True, org.label-schema.build-date=20240307, io.buildah.version=1.33.5) 2024-03-24T11:35:01.720 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:35:01 smithi062 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.1.service: Deactivated successfully. 2024-03-24T11:35:01.720 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:35:01 smithi062 systemd[1]: Stopped Ceph osd.1 for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:35:01.720 INFO:journalctl@ceph.osd.1.smithi062.stdout:Mar 24 11:35:01 smithi062 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.1.service: Consumed 7.521s CPU time. 2024-03-24T11:35:01.737 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T11:35:01.737 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-24T11:35:01.737 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-24T11:35:01.737 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.2 2024-03-24T11:35:01.977 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:35:01 smithi062 systemd[1]: Stopping Ceph osd.2 for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:35:02.477 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:35:02 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2[137500]: 2024-03-24T11:35:02.063+0000 7ff63ade1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T11:35:02.477 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:35:02 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2[137500]: 2024-03-24T11:35:02.063+0000 7ff63ade1700 -1 osd.2 180 *** Got signal Terminated *** 2024-03-24T11:35:02.477 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:35:02 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2[137500]: 2024-03-24T11:35:02.063+0000 7ff63ade1700 -1 osd.2 180 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T11:35:03.727 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:35:03 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[64072]: level=error ts=2024-03-24T11:35:03.555Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=6 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi139.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.139:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi062.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.62:8443: connect: connection refused" 2024-03-24T11:35:03.727 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:35:03 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[64072]: level=warn ts=2024-03-24T11:35:03.556Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi062.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.62:8443: connect: connection refused" 2024-03-24T11:35:03.727 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:35:03 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[64072]: level=warn ts=2024-03-24T11:35:03.556Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi139.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.139:8443: connect: connection refused" 2024-03-24T11:35:07.674 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:35:07 smithi062 podman[181328]: 2024-03-24 11:35:07.398581388 +0000 UTC m=+5.431009770 container died 0eba40dc96b39a58e478f1b7059d1ef6041365fca7f5c1e4949e69c5807894dd (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-24T11:35:07.977 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:35:07 smithi062 podman[181328]: 2024-03-24 11:35:07.848063855 +0000 UTC m=+5.880492243 container cleanup 0eba40dc96b39a58e478f1b7059d1ef6041365fca7f5c1e4949e69c5807894dd (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-db0330b, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T11:35:07.977 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:35:07 smithi062 bash[181328]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2 2024-03-24T11:35:08.457 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:35:08 smithi062 podman[181345]: 2024-03-24 11:35:08.041433901 +0000 UTC m=+0.632232306 container remove 0eba40dc96b39a58e478f1b7059d1ef6041365fca7f5c1e4949e69c5807894dd (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-db0330b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True) 2024-03-24T11:35:08.727 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:35:08 smithi062 podman[181434]: 2024-03-24 11:35:08.360889043 +0000 UTC m=+0.017928938 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:35:08.727 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:35:08 smithi062 podman[181434]: 2024-03-24 11:35:08.467221919 +0000 UTC m=+0.124261851 container create 4163ec64665fc0246df20409499367604af5dc6f628c1f7356a7ab2581dc1407 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-db0330b, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux ) 2024-03-24T11:35:08.727 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:35:08 smithi062 podman[181434]: 2024-03-24 11:35:08.694477123 +0000 UTC m=+0.351517022 container init 4163ec64665fc0246df20409499367604af5dc6f628c1f7356a7ab2581dc1407 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-24T11:35:08.727 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:35:08 smithi062 podman[181434]: 2024-03-24 11:35:08.697518246 +0000 UTC m=+0.354558137 container start 4163ec64665fc0246df20409499367604af5dc6f628c1f7356a7ab2581dc1407 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240307, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-24T11:35:09.227 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:35:08 smithi062 podman[181434]: 2024-03-24 11:35:08.786201164 +0000 UTC m=+0.443241061 container attach 4163ec64665fc0246df20409499367604af5dc6f628c1f7356a7ab2581dc1407 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-24T11:35:09.227 INFO:journalctl@ceph.osd.2.smithi062.stdout:Mar 24 11:35:08 smithi062 podman[181434]: 2024-03-24 11:35:08.994375836 +0000 UTC m=+0.651415736 container died 4163ec64665fc0246df20409499367604af5dc6f628c1f7356a7ab2581dc1407 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-db0330b, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2) 2024-03-24T11:35:09.589 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.2.service' 2024-03-24T11:35:09.634 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T11:35:09.634 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-24T11:35:09.634 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-24T11:35:09.635 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.3 2024-03-24T11:35:09.967 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:35:09 smithi062 systemd[1]: Stopping Ceph osd.3 for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:35:10.227 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:35:09 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3[149887]: 2024-03-24T11:35:09.966+0000 7fccbf16f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T11:35:10.229 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:35:09 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3[149887]: 2024-03-24T11:35:09.966+0000 7fccbf16f700 -1 osd.3 180 *** Got signal Terminated *** 2024-03-24T11:35:10.230 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:35:09 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3[149887]: 2024-03-24T11:35:09.966+0000 7fccbf16f700 -1 osd.3 180 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T11:35:13.327 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:35:13 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3[149887]: 2024-03-24T11:35:13.020+0000 7fccb775b700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.374095+0000 front 2024-03-24T11:34:47.374052+0000 (oldest deadline 2024-03-24T11:35:12.674116+0000) 2024-03-24T11:35:13.558 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:13.261+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.556704+0000 front 2024-03-24T11:34:47.556794+0000 (oldest deadline 2024-03-24T11:35:12.856557+0000) 2024-03-24T11:35:13.727 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:35:13 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[64072]: level=error ts=2024-03-24T11:35:13.555Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=6 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi139.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.139:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi062.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.62:8443: connect: connection refused" 2024-03-24T11:35:13.727 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:35:13 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[64072]: level=warn ts=2024-03-24T11:35:13.556Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi062.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.62:8443: connect: connection refused" 2024-03-24T11:35:13.727 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:35:13 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[64072]: level=warn ts=2024-03-24T11:35:13.557Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi139.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.139:8443: connect: connection refused" 2024-03-24T11:35:13.817 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:13 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:13.556+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.222394+0000 front 2024-03-24T11:34:47.222378+0000 (oldest deadline 2024-03-24T11:35:13.122152+0000) 2024-03-24T11:35:14.226 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:35:14 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3[149887]: 2024-03-24T11:35:14.027+0000 7fccb775b700 -1 osd.3 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.374095+0000 front 2024-03-24T11:34:47.374052+0000 (oldest deadline 2024-03-24T11:35:12.674116+0000) 2024-03-24T11:35:14.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:14 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:14.220+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.556704+0000 front 2024-03-24T11:34:47.556794+0000 (oldest deadline 2024-03-24T11:35:12.856557+0000) 2024-03-24T11:35:14.856 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:14 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:14.532+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.222394+0000 front 2024-03-24T11:34:47.222378+0000 (oldest deadline 2024-03-24T11:35:13.122152+0000) 2024-03-24T11:35:15.522 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:35:15 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5[132460]: 2024-03-24T11:35:15.307+0000 7f4303807700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:49.193180+0000 front 2024-03-24T11:34:49.193126+0000 (oldest deadline 2024-03-24T11:35:15.093106+0000) 2024-03-24T11:35:15.522 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:15 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:15.205+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.556704+0000 front 2024-03-24T11:34:47.556794+0000 (oldest deadline 2024-03-24T11:35:12.856557+0000) 2024-03-24T11:35:15.605 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:35:15 smithi062 podman[181538]: 2024-03-24 11:35:15.324272647 +0000 UTC m=+5.468097361 container died 7501d2e4c718608a078f33a6d3f3096aa4bdb785d93f758443445ba420af4f77 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-24T11:35:15.858 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:15 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:15.564+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.222394+0000 front 2024-03-24T11:34:47.222378+0000 (oldest deadline 2024-03-24T11:35:13.122152+0000) 2024-03-24T11:35:15.858 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:35:15 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:35:15.522Z caller=refresh.go:98 level=error component="discovery manager scrape" discovery=http msg="Unable to refresh target groups" err="Get \"http://172.21.15.62:8765/sd/prometheus/sd-config?service=node-exporter\": dial tcp 172.21.15.62:8765: connect: connection refused" 2024-03-24T11:35:15.859 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:35:15 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:35:15.524Z caller=refresh.go:98 level=error component="discovery manager notify" discovery=http msg="Unable to refresh target groups" err="Get \"http://172.21.15.62:8765/sd/prometheus/sd-config?service=alertmanager\": dial tcp 172.21.15.62:8765: connect: connection refused" 2024-03-24T11:35:15.859 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:35:15 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:35:15.524Z caller=refresh.go:98 level=error component="discovery manager scrape" discovery=http msg="Unable to refresh target groups" err="Get \"http://172.21.15.62:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.62:8765: connect: connection refused" 2024-03-24T11:35:15.859 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:35:15 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:35:15.524Z caller=refresh.go:98 level=error component="discovery manager scrape" discovery=http msg="Unable to refresh target groups" err="Get \"http://172.21.15.62:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.62:8765: connect: connection refused" 2024-03-24T11:35:15.938 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:35:15 smithi062 podman[181538]: 2024-03-24 11:35:15.703930861 +0000 UTC m=+5.847755574 container cleanup 7501d2e4c718608a078f33a6d3f3096aa4bdb785d93f758443445ba420af4f77 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3, RELEASE=squid-db0330b, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-24T11:35:15.939 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:35:15 smithi062 bash[181538]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3 2024-03-24T11:35:16.226 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:35:15 smithi062 podman[181668]: 2024-03-24 11:35:15.93843948 +0000 UTC m=+0.611239642 container remove 7501d2e4c718608a078f33a6d3f3096aa4bdb785d93f758443445ba420af4f77 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-24T11:35:16.555 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:35:16 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4[127378]: 2024-03-24T11:35:16.272+0000 7f09160f3700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:45.614585+0000 front 2024-03-24T11:34:45.614654+0000 (oldest deadline 2024-03-24T11:35:15.614827+0000) 2024-03-24T11:35:16.555 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:35:16 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5[132460]: 2024-03-24T11:35:16.268+0000 7f4303807700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:49.193180+0000 front 2024-03-24T11:34:49.193126+0000 (oldest deadline 2024-03-24T11:35:15.093106+0000) 2024-03-24T11:35:16.555 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:16 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:16.214+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.556704+0000 front 2024-03-24T11:34:47.556794+0000 (oldest deadline 2024-03-24T11:35:12.856557+0000) 2024-03-24T11:35:16.592 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:35:16 smithi062 podman[181762]: 2024-03-24 11:35:16.178230345 +0000 UTC m=+0.017258284 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:35:16.592 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:35:16 smithi062 podman[181762]: 2024-03-24 11:35:16.289073987 +0000 UTC m=+0.128101925 container create d7d3d9261919820ddf2b6523137e270f11451e41a22bb67bde905e8987c41735 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5) 2024-03-24T11:35:16.592 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:35:16 smithi062 podman[181762]: 2024-03-24 11:35:16.516772962 +0000 UTC m=+0.355800901 container init d7d3d9261919820ddf2b6523137e270f11451e41a22bb67bde905e8987c41735 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.build-date=20240307, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-24T11:35:16.592 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:35:16 smithi062 podman[181762]: 2024-03-24 11:35:16.51979613 +0000 UTC m=+0.358824068 container start d7d3d9261919820ddf2b6523137e270f11451e41a22bb67bde905e8987c41735 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3-deactivate, RELEASE=squid-db0330b, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-24T11:35:16.592 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:35:16 smithi062 podman[181762]: 2024-03-24 11:35:16.591746544 +0000 UTC m=+0.430774485 container attach d7d3d9261919820ddf2b6523137e270f11451e41a22bb67bde905e8987c41735 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3-deactivate, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T11:35:16.858 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:16 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:16.558+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.222394+0000 front 2024-03-24T11:34:47.222378+0000 (oldest deadline 2024-03-24T11:35:13.122152+0000) 2024-03-24T11:35:16.978 INFO:journalctl@ceph.osd.3.smithi062.stdout:Mar 24 11:35:16 smithi062 podman[181762]: 2024-03-24 11:35:16.836575219 +0000 UTC m=+0.675603153 container died d7d3d9261919820ddf2b6523137e270f11451e41a22bb67bde905e8987c41735 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-db0330b, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T11:35:17.434 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.3.service' 2024-03-24T11:35:17.477 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T11:35:17.478 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-24T11:35:17.478 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-24T11:35:17.478 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.4 2024-03-24T11:35:17.510 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:35:17 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4[127378]: 2024-03-24T11:35:17.321+0000 7f09160f3700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:45.614585+0000 front 2024-03-24T11:34:45.614654+0000 (oldest deadline 2024-03-24T11:35:15.614827+0000) 2024-03-24T11:35:17.510 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:35:17 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5[132460]: 2024-03-24T11:35:17.310+0000 7f4303807700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:49.193180+0000 front 2024-03-24T11:34:49.193126+0000 (oldest deadline 2024-03-24T11:35:15.093106+0000) 2024-03-24T11:35:17.511 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:17 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:17.239+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.556704+0000 front 2024-03-24T11:34:47.556794+0000 (oldest deadline 2024-03-24T11:35:12.856557+0000) 2024-03-24T11:35:17.791 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:35:17 smithi139 systemd[1]: Stopping Ceph osd.4 for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:35:17.791 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:17 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:17.514+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.222394+0000 front 2024-03-24T11:34:47.222378+0000 (oldest deadline 2024-03-24T11:35:13.122152+0000) 2024-03-24T11:35:18.108 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:35:17 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4[127378]: 2024-03-24T11:35:17.790+0000 7f091db07700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T11:35:18.108 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:35:17 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4[127378]: 2024-03-24T11:35:17.790+0000 7f091db07700 -1 osd.4 180 *** Got signal Terminated *** 2024-03-24T11:35:18.108 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:35:17 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4[127378]: 2024-03-24T11:35:17.790+0000 7f091db07700 -1 osd.4 180 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T11:35:18.555 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:35:18 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4[127378]: 2024-03-24T11:35:18.329+0000 7f09160f3700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:45.614585+0000 front 2024-03-24T11:34:45.614654+0000 (oldest deadline 2024-03-24T11:35:15.614827+0000) 2024-03-24T11:35:18.555 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:35:18 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5[132460]: 2024-03-24T11:35:18.340+0000 7f4303807700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:49.193180+0000 front 2024-03-24T11:34:49.193126+0000 (oldest deadline 2024-03-24T11:35:15.093106+0000) 2024-03-24T11:35:18.555 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:18 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:18.214+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.556704+0000 front 2024-03-24T11:34:47.556794+0000 (oldest deadline 2024-03-24T11:35:12.856557+0000) 2024-03-24T11:35:18.858 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:18 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:18.553+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.222394+0000 front 2024-03-24T11:34:47.222378+0000 (oldest deadline 2024-03-24T11:35:13.122152+0000) 2024-03-24T11:35:19.510 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:35:19 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4[127378]: 2024-03-24T11:35:19.340+0000 7f09160f3700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:45.614585+0000 front 2024-03-24T11:34:45.614654+0000 (oldest deadline 2024-03-24T11:35:15.614827+0000) 2024-03-24T11:35:19.510 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:35:19 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5[132460]: 2024-03-24T11:35:19.302+0000 7f4303807700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:49.193180+0000 front 2024-03-24T11:34:49.193126+0000 (oldest deadline 2024-03-24T11:35:15.093106+0000) 2024-03-24T11:35:19.510 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:19 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:19.216+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.556704+0000 front 2024-03-24T11:34:47.556794+0000 (oldest deadline 2024-03-24T11:35:12.856557+0000) 2024-03-24T11:35:19.858 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:19 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:19.509+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.222394+0000 front 2024-03-24T11:34:47.222378+0000 (oldest deadline 2024-03-24T11:35:13.122152+0000) 2024-03-24T11:35:20.515 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:35:20 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4[127378]: 2024-03-24T11:35:20.341+0000 7f09160f3700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:45.614585+0000 front 2024-03-24T11:34:45.614654+0000 (oldest deadline 2024-03-24T11:35:15.614827+0000) 2024-03-24T11:35:20.515 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:35:20 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5[132460]: 2024-03-24T11:35:20.267+0000 7f4303807700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:49.193180+0000 front 2024-03-24T11:34:49.193126+0000 (oldest deadline 2024-03-24T11:35:15.093106+0000) 2024-03-24T11:35:20.515 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:20 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:20.175+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.556704+0000 front 2024-03-24T11:34:47.556794+0000 (oldest deadline 2024-03-24T11:35:12.856557+0000) 2024-03-24T11:35:20.858 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:20 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:20.514+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.222394+0000 front 2024-03-24T11:34:47.222378+0000 (oldest deadline 2024-03-24T11:35:13.122152+0000) 2024-03-24T11:35:20.858 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:20 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:20.514+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:57.222996+0000 front 2024-03-24T11:34:57.223035+0000 (oldest deadline 2024-03-24T11:35:20.122977+0000) 2024-03-24T11:35:21.502 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:35:21 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4[127378]: 2024-03-24T11:35:21.362+0000 7f09160f3700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:45.614585+0000 front 2024-03-24T11:34:45.614654+0000 (oldest deadline 2024-03-24T11:35:15.614827+0000) 2024-03-24T11:35:21.502 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:35:21 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5[132460]: 2024-03-24T11:35:21.254+0000 7f4303807700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:49.193180+0000 front 2024-03-24T11:34:49.193126+0000 (oldest deadline 2024-03-24T11:35:15.093106+0000) 2024-03-24T11:35:21.502 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:35:21 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5[132460]: 2024-03-24T11:35:21.254+0000 7f4303807700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:55.093571+0000 front 2024-03-24T11:34:55.093633+0000 (oldest deadline 2024-03-24T11:35:20.993534+0000) 2024-03-24T11:35:21.503 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:21 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:21.185+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.556704+0000 front 2024-03-24T11:34:47.556794+0000 (oldest deadline 2024-03-24T11:35:12.856557+0000) 2024-03-24T11:35:21.503 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:21 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:21.185+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:58.657782+0000 front 2024-03-24T11:34:58.657834+0000 (oldest deadline 2024-03-24T11:35:20.357736+0000) 2024-03-24T11:35:21.858 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:21 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:21.501+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.222394+0000 front 2024-03-24T11:34:47.222378+0000 (oldest deadline 2024-03-24T11:35:13.122152+0000) 2024-03-24T11:35:21.858 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:21 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:21.501+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:57.222996+0000 front 2024-03-24T11:34:57.223035+0000 (oldest deadline 2024-03-24T11:35:20.122977+0000) 2024-03-24T11:35:22.508 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:35:22 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4[127378]: 2024-03-24T11:35:22.340+0000 7f09160f3700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:45.614585+0000 front 2024-03-24T11:34:45.614654+0000 (oldest deadline 2024-03-24T11:35:15.614827+0000) 2024-03-24T11:35:22.509 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:35:22 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4[127378]: 2024-03-24T11:35:22.340+0000 7f09160f3700 -1 osd.4 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:56.715702+0000 front 2024-03-24T11:34:56.715755+0000 (oldest deadline 2024-03-24T11:35:22.015623+0000) 2024-03-24T11:35:22.509 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:35:22 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5[132460]: 2024-03-24T11:35:22.254+0000 7f4303807700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:49.193180+0000 front 2024-03-24T11:34:49.193126+0000 (oldest deadline 2024-03-24T11:35:15.093106+0000) 2024-03-24T11:35:22.509 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:35:22 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5[132460]: 2024-03-24T11:35:22.254+0000 7f4303807700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:55.093571+0000 front 2024-03-24T11:34:55.093633+0000 (oldest deadline 2024-03-24T11:35:20.993534+0000) 2024-03-24T11:35:22.509 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:22 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:22.220+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.556704+0000 front 2024-03-24T11:34:47.556794+0000 (oldest deadline 2024-03-24T11:35:12.856557+0000) 2024-03-24T11:35:22.509 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:22 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:22.220+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:58.657782+0000 front 2024-03-24T11:34:58.657834+0000 (oldest deadline 2024-03-24T11:35:20.357736+0000) 2024-03-24T11:35:22.858 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:22 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:22.508+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.222394+0000 front 2024-03-24T11:34:47.222378+0000 (oldest deadline 2024-03-24T11:35:13.122152+0000) 2024-03-24T11:35:22.858 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:22 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:22.508+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:57.222996+0000 front 2024-03-24T11:34:57.223035+0000 (oldest deadline 2024-03-24T11:35:20.122977+0000) 2024-03-24T11:35:23.425 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:35:23 smithi139 podman[155793]: 2024-03-24 11:35:23.130837969 +0000 UTC m=+5.449188805 container died 5d0f7f05ec6d71fd9afc6d855efcf50fcde6399b260f23dada7ef82d4bd36f58 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, ceph=True, CEPH_POINT_RELEASE=) 2024-03-24T11:35:23.425 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:35:23 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5[132460]: 2024-03-24T11:35:23.230+0000 7f4303807700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:49.193180+0000 front 2024-03-24T11:34:49.193126+0000 (oldest deadline 2024-03-24T11:35:15.093106+0000) 2024-03-24T11:35:23.425 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:35:23 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5[132460]: 2024-03-24T11:35:23.230+0000 7f4303807700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:55.093571+0000 front 2024-03-24T11:34:55.093633+0000 (oldest deadline 2024-03-24T11:35:20.993534+0000) 2024-03-24T11:35:23.426 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:23 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:23.179+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.556704+0000 front 2024-03-24T11:34:47.556794+0000 (oldest deadline 2024-03-24T11:35:12.856557+0000) 2024-03-24T11:35:23.426 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:23 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:23.179+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:58.657782+0000 front 2024-03-24T11:34:58.657834+0000 (oldest deadline 2024-03-24T11:35:20.357736+0000) 2024-03-24T11:35:23.858 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:35:23 smithi139 podman[155793]: 2024-03-24 11:35:23.649408452 +0000 UTC m=+5.967759285 container cleanup 5d0f7f05ec6d71fd9afc6d855efcf50fcde6399b260f23dada7ef82d4bd36f58 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-db0330b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T11:35:23.858 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:35:23 smithi139 bash[155793]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4 2024-03-24T11:35:23.858 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:23 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:23.531+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.222394+0000 front 2024-03-24T11:34:47.222378+0000 (oldest deadline 2024-03-24T11:35:13.122152+0000) 2024-03-24T11:35:23.858 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:23 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:23.531+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:57.222996+0000 front 2024-03-24T11:34:57.223035+0000 (oldest deadline 2024-03-24T11:35:20.122977+0000) 2024-03-24T11:35:23.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:35:23 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[64072]: level=error ts=2024-03-24T11:35:23.556Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=6 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi139.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.139:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi062.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.62:8443: connect: connection refused" 2024-03-24T11:35:23.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:35:23 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[64072]: level=warn ts=2024-03-24T11:35:23.557Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi062.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.62:8443: connect: connection refused" 2024-03-24T11:35:23.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:35:23 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[64072]: level=warn ts=2024-03-24T11:35:23.558Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi139.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.139:8443: connect: connection refused" 2024-03-24T11:35:24.182 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:35:23 smithi139 podman[155812]: 2024-03-24 11:35:23.867387935 +0000 UTC m=+0.734841319 container remove 5d0f7f05ec6d71fd9afc6d855efcf50fcde6399b260f23dada7ef82d4bd36f58 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-db0330b, org.label-schema.license=GPLv2) 2024-03-24T11:35:24.492 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:35:24 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5[132460]: 2024-03-24T11:35:24.181+0000 7f4303807700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:49.193180+0000 front 2024-03-24T11:34:49.193126+0000 (oldest deadline 2024-03-24T11:35:15.093106+0000) 2024-03-24T11:35:24.493 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:35:24 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5[132460]: 2024-03-24T11:35:24.181+0000 7f4303807700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:55.093571+0000 front 2024-03-24T11:34:55.093633+0000 (oldest deadline 2024-03-24T11:35:20.993534+0000) 2024-03-24T11:35:24.493 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:24 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:24.211+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.556704+0000 front 2024-03-24T11:34:47.556794+0000 (oldest deadline 2024-03-24T11:35:12.856557+0000) 2024-03-24T11:35:24.493 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:24 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:24.211+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:58.657782+0000 front 2024-03-24T11:34:58.657834+0000 (oldest deadline 2024-03-24T11:35:20.357736+0000) 2024-03-24T11:35:24.493 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:35:24 smithi139 podman[155902]: 2024-03-24 11:35:24.170975199 +0000 UTC m=+0.017927479 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:35:24.493 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:35:24 smithi139 podman[155902]: 2024-03-24 11:35:24.285018589 +0000 UTC m=+0.131970850 container create 2451758701d8157fc87718e9e4c505a25e97f7a262894740b052ed491395deb8 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307) 2024-03-24T11:35:24.820 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:24 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:24.492+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.222394+0000 front 2024-03-24T11:34:47.222378+0000 (oldest deadline 2024-03-24T11:35:13.122152+0000) 2024-03-24T11:35:24.820 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:24 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:24.492+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:57.222996+0000 front 2024-03-24T11:34:57.223035+0000 (oldest deadline 2024-03-24T11:35:20.122977+0000) 2024-03-24T11:35:24.820 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:35:24 smithi139 podman[155902]: 2024-03-24 11:35:24.526850406 +0000 UTC m=+0.373802684 container init 2451758701d8157fc87718e9e4c505a25e97f7a262894740b052ed491395deb8 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T11:35:24.820 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:35:24 smithi139 podman[155902]: 2024-03-24 11:35:24.530512347 +0000 UTC m=+0.377464612 container start 2451758701d8157fc87718e9e4c505a25e97f7a262894740b052ed491395deb8 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-24T11:35:24.820 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:35:24 smithi139 podman[155902]: 2024-03-24 11:35:24.60432258 +0000 UTC m=+0.451274851 container attach 2451758701d8157fc87718e9e4c505a25e97f7a262894740b052ed491395deb8 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b) 2024-03-24T11:35:25.108 INFO:journalctl@ceph.osd.4.smithi139.stdout:Mar 24 11:35:24 smithi139 podman[155902]: 2024-03-24 11:35:24.820148776 +0000 UTC m=+0.667101029 container died 2451758701d8157fc87718e9e4c505a25e97f7a262894740b052ed491395deb8 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, RELEASE=squid-db0330b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-24T11:35:25.458 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:35:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5[132460]: 2024-03-24T11:35:25.199+0000 7f4303807700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:49.193180+0000 front 2024-03-24T11:34:49.193126+0000 (oldest deadline 2024-03-24T11:35:15.093106+0000) 2024-03-24T11:35:25.458 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:35:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5[132460]: 2024-03-24T11:35:25.199+0000 7f4303807700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:55.093571+0000 front 2024-03-24T11:34:55.093633+0000 (oldest deadline 2024-03-24T11:35:20.993534+0000) 2024-03-24T11:35:25.458 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:25.169+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.556704+0000 front 2024-03-24T11:34:47.556794+0000 (oldest deadline 2024-03-24T11:35:12.856557+0000) 2024-03-24T11:35:25.458 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:25.169+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:58.657782+0000 front 2024-03-24T11:34:58.657834+0000 (oldest deadline 2024-03-24T11:35:20.357736+0000) 2024-03-24T11:35:25.476 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.4.service' 2024-03-24T11:35:25.561 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T11:35:25.561 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-24T11:35:25.561 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-24T11:35:25.561 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.5 2024-03-24T11:35:25.858 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:35:25 smithi139 systemd[1]: Stopping Ceph osd.5 for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:35:25.858 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:25.489+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.222394+0000 front 2024-03-24T11:34:47.222378+0000 (oldest deadline 2024-03-24T11:35:13.122152+0000) 2024-03-24T11:35:25.858 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:25.489+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:57.222996+0000 front 2024-03-24T11:34:57.223035+0000 (oldest deadline 2024-03-24T11:35:20.122977+0000) 2024-03-24T11:35:26.190 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:35:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5[132460]: 2024-03-24T11:35:25.910+0000 7f4307213700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T11:35:26.191 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:35:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5[132460]: 2024-03-24T11:35:25.910+0000 7f4307213700 -1 osd.5 180 *** Got signal Terminated *** 2024-03-24T11:35:26.191 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:35:25 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5[132460]: 2024-03-24T11:35:25.910+0000 7f4307213700 -1 osd.5 180 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T11:35:26.536 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:26 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:26.190+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.556704+0000 front 2024-03-24T11:34:47.556794+0000 (oldest deadline 2024-03-24T11:35:12.856557+0000) 2024-03-24T11:35:26.536 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:26 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:26.190+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:58.657782+0000 front 2024-03-24T11:34:58.657834+0000 (oldest deadline 2024-03-24T11:35:20.357736+0000) 2024-03-24T11:35:26.536 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:35:26 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5[132460]: 2024-03-24T11:35:26.202+0000 7f4303807700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:49.193180+0000 front 2024-03-24T11:34:49.193126+0000 (oldest deadline 2024-03-24T11:35:15.093106+0000) 2024-03-24T11:35:26.536 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:35:26 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5[132460]: 2024-03-24T11:35:26.202+0000 7f4303807700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:55.093571+0000 front 2024-03-24T11:34:55.093633+0000 (oldest deadline 2024-03-24T11:35:20.993534+0000) 2024-03-24T11:35:26.858 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:26 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:26.536+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.222394+0000 front 2024-03-24T11:34:47.222378+0000 (oldest deadline 2024-03-24T11:35:13.122152+0000) 2024-03-24T11:35:26.858 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:26 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:26.536+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:57.222996+0000 front 2024-03-24T11:34:57.223035+0000 (oldest deadline 2024-03-24T11:35:20.122977+0000) 2024-03-24T11:35:27.515 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:35:27 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5[132460]: 2024-03-24T11:35:27.207+0000 7f4303807700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:49.193180+0000 front 2024-03-24T11:34:49.193126+0000 (oldest deadline 2024-03-24T11:35:15.093106+0000) 2024-03-24T11:35:27.516 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:35:27 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5[132460]: 2024-03-24T11:35:27.207+0000 7f4303807700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:55.093571+0000 front 2024-03-24T11:34:55.093633+0000 (oldest deadline 2024-03-24T11:35:20.993534+0000) 2024-03-24T11:35:27.516 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:27 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:27.153+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.556704+0000 front 2024-03-24T11:34:47.556794+0000 (oldest deadline 2024-03-24T11:35:12.856557+0000) 2024-03-24T11:35:27.516 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:27 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:27.153+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:58.657782+0000 front 2024-03-24T11:34:58.657834+0000 (oldest deadline 2024-03-24T11:35:20.357736+0000) 2024-03-24T11:35:27.858 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:27 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:27.515+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.222394+0000 front 2024-03-24T11:34:47.222378+0000 (oldest deadline 2024-03-24T11:35:13.122152+0000) 2024-03-24T11:35:27.858 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:27 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:27.515+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:57.222996+0000 front 2024-03-24T11:34:57.223035+0000 (oldest deadline 2024-03-24T11:35:20.122977+0000) 2024-03-24T11:35:28.504 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:28 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:28.133+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.556704+0000 front 2024-03-24T11:34:47.556794+0000 (oldest deadline 2024-03-24T11:35:12.856557+0000) 2024-03-24T11:35:28.504 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:28 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:28.133+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:58.657782+0000 front 2024-03-24T11:34:58.657834+0000 (oldest deadline 2024-03-24T11:35:20.357736+0000) 2024-03-24T11:35:28.504 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:35:28 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5[132460]: 2024-03-24T11:35:28.230+0000 7f4303807700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:49.193180+0000 front 2024-03-24T11:34:49.193126+0000 (oldest deadline 2024-03-24T11:35:15.093106+0000) 2024-03-24T11:35:28.504 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:35:28 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5[132460]: 2024-03-24T11:35:28.230+0000 7f4303807700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:55.093571+0000 front 2024-03-24T11:34:55.093633+0000 (oldest deadline 2024-03-24T11:35:20.993534+0000) 2024-03-24T11:35:28.858 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:28 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:28.503+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.222394+0000 front 2024-03-24T11:34:47.222378+0000 (oldest deadline 2024-03-24T11:35:13.122152+0000) 2024-03-24T11:35:28.858 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:28 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:28.503+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:57.222996+0000 front 2024-03-24T11:34:57.223035+0000 (oldest deadline 2024-03-24T11:35:20.122977+0000) 2024-03-24T11:35:29.455 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:35:29 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5[132460]: 2024-03-24T11:35:29.191+0000 7f4303807700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:49.193180+0000 front 2024-03-24T11:34:49.193126+0000 (oldest deadline 2024-03-24T11:35:15.093106+0000) 2024-03-24T11:35:29.455 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:35:29 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5[132460]: 2024-03-24T11:35:29.191+0000 7f4303807700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:55.093571+0000 front 2024-03-24T11:34:55.093633+0000 (oldest deadline 2024-03-24T11:35:20.993534+0000) 2024-03-24T11:35:29.457 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:29 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:29.123+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.556704+0000 front 2024-03-24T11:34:47.556794+0000 (oldest deadline 2024-03-24T11:35:12.856557+0000) 2024-03-24T11:35:29.457 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:29 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:29.123+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:58.657782+0000 front 2024-03-24T11:34:58.657834+0000 (oldest deadline 2024-03-24T11:35:20.357736+0000) 2024-03-24T11:35:29.858 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:29 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:29.454+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.222394+0000 front 2024-03-24T11:34:47.222378+0000 (oldest deadline 2024-03-24T11:35:13.122152+0000) 2024-03-24T11:35:29.858 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:29 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:29.454+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:57.222996+0000 front 2024-03-24T11:34:57.223035+0000 (oldest deadline 2024-03-24T11:35:20.122977+0000) 2024-03-24T11:35:30.233 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:30 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:30.156+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.556704+0000 front 2024-03-24T11:34:47.556794+0000 (oldest deadline 2024-03-24T11:35:12.856557+0000) 2024-03-24T11:35:30.233 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:30 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:30.156+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:58.657782+0000 front 2024-03-24T11:34:58.657834+0000 (oldest deadline 2024-03-24T11:35:20.357736+0000) 2024-03-24T11:35:30.233 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:30 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:30.156+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6822 osd.2 since back 2024-03-24T11:35:06.258940+0000 front 2024-03-24T11:35:06.258952+0000 (oldest deadline 2024-03-24T11:35:29.758769+0000) 2024-03-24T11:35:30.498 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:35:30 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5[132460]: 2024-03-24T11:35:30.232+0000 7f4303807700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:49.193180+0000 front 2024-03-24T11:34:49.193126+0000 (oldest deadline 2024-03-24T11:35:15.093106+0000) 2024-03-24T11:35:30.499 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:35:30 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5[132460]: 2024-03-24T11:35:30.232+0000 7f4303807700 -1 osd.5 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:55.093571+0000 front 2024-03-24T11:34:55.093633+0000 (oldest deadline 2024-03-24T11:35:20.993534+0000) 2024-03-24T11:35:30.858 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:30 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:30.497+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.222394+0000 front 2024-03-24T11:34:47.222378+0000 (oldest deadline 2024-03-24T11:35:13.122152+0000) 2024-03-24T11:35:30.858 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:30 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:30.497+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:57.222996+0000 front 2024-03-24T11:34:57.223035+0000 (oldest deadline 2024-03-24T11:35:20.122977+0000) 2024-03-24T11:35:31.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:31 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:31.128+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.556704+0000 front 2024-03-24T11:34:47.556794+0000 (oldest deadline 2024-03-24T11:35:12.856557+0000) 2024-03-24T11:35:31.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:31 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:31.128+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:58.657782+0000 front 2024-03-24T11:34:58.657834+0000 (oldest deadline 2024-03-24T11:35:20.357736+0000) 2024-03-24T11:35:31.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:31 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:31.128+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6822 osd.2 since back 2024-03-24T11:35:06.258940+0000 front 2024-03-24T11:35:06.258952+0000 (oldest deadline 2024-03-24T11:35:29.758769+0000) 2024-03-24T11:35:31.858 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:31 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:31.540+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.222394+0000 front 2024-03-24T11:34:47.222378+0000 (oldest deadline 2024-03-24T11:35:13.122152+0000) 2024-03-24T11:35:31.858 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:31 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:31.540+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:57.222996+0000 front 2024-03-24T11:34:57.223035+0000 (oldest deadline 2024-03-24T11:35:20.122977+0000) 2024-03-24T11:35:31.858 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:31 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:31.540+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6822 osd.2 since back 2024-03-24T11:35:06.024102+0000 front 2024-03-24T11:35:06.024093+0000 (oldest deadline 2024-03-24T11:35:31.323890+0000) 2024-03-24T11:35:32.136 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:35:31 smithi139 podman[156009]: 2024-03-24 11:35:31.867041013 +0000 UTC m=+6.078181974 container died 4ea95bfd21272c31051c4598267942aabd9b2ab886bf57b40f01455af009da8c (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-db0330b) 2024-03-24T11:35:32.517 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:35:32 smithi139 podman[156009]: 2024-03-24 11:35:32.356043663 +0000 UTC m=+6.567184609 container cleanup 4ea95bfd21272c31051c4598267942aabd9b2ab886bf57b40f01455af009da8c (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5, org.label-schema.license=GPLv2, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307) 2024-03-24T11:35:32.517 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:35:32 smithi139 bash[156009]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5 2024-03-24T11:35:32.518 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:32 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:32.136+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.556704+0000 front 2024-03-24T11:34:47.556794+0000 (oldest deadline 2024-03-24T11:35:12.856557+0000) 2024-03-24T11:35:32.518 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:32 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:32.136+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:58.657782+0000 front 2024-03-24T11:34:58.657834+0000 (oldest deadline 2024-03-24T11:35:20.357736+0000) 2024-03-24T11:35:32.518 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:32 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:32.136+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6822 osd.2 since back 2024-03-24T11:35:06.258940+0000 front 2024-03-24T11:35:06.258952+0000 (oldest deadline 2024-03-24T11:35:29.758769+0000) 2024-03-24T11:35:32.809 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:35:32 smithi139 podman[156361]: 2024-03-24 11:35:32.57418248 +0000 UTC m=+1.332380535 container remove 4ea95bfd21272c31051c4598267942aabd9b2ab886bf57b40f01455af009da8c (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5, GIT_CLEAN=True, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-db0330b, org.label-schema.schema-version=1.0) 2024-03-24T11:35:32.809 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:32 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:32.516+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.222394+0000 front 2024-03-24T11:34:47.222378+0000 (oldest deadline 2024-03-24T11:35:13.122152+0000) 2024-03-24T11:35:32.809 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:32 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:32.516+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:57.222996+0000 front 2024-03-24T11:34:57.223035+0000 (oldest deadline 2024-03-24T11:35:20.122977+0000) 2024-03-24T11:35:32.809 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:32 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:32.516+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6822 osd.2 since back 2024-03-24T11:35:06.024102+0000 front 2024-03-24T11:35:06.024093+0000 (oldest deadline 2024-03-24T11:35:31.323890+0000) 2024-03-24T11:35:33.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:33 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:33.089+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.556704+0000 front 2024-03-24T11:34:47.556794+0000 (oldest deadline 2024-03-24T11:35:12.856557+0000) 2024-03-24T11:35:33.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:33 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:33.089+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:58.657782+0000 front 2024-03-24T11:34:58.657834+0000 (oldest deadline 2024-03-24T11:35:20.357736+0000) 2024-03-24T11:35:33.359 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:33 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:33.089+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6822 osd.2 since back 2024-03-24T11:35:06.258940+0000 front 2024-03-24T11:35:06.258952+0000 (oldest deadline 2024-03-24T11:35:29.758769+0000) 2024-03-24T11:35:33.727 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:35:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[64072]: level=error ts=2024-03-24T11:35:33.557Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=6 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi062.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.62:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi139.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.139:8443: connect: connection refused" 2024-03-24T11:35:33.727 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:35:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[64072]: level=warn ts=2024-03-24T11:35:33.558Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi062.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.62:8443: connect: connection refused" 2024-03-24T11:35:33.727 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:35:33 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[64072]: level=warn ts=2024-03-24T11:35:33.559Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi139.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.139:8443: connect: connection refused" 2024-03-24T11:35:33.858 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:33 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:33.556+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.222394+0000 front 2024-03-24T11:34:47.222378+0000 (oldest deadline 2024-03-24T11:35:13.122152+0000) 2024-03-24T11:35:33.858 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:33 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:33.556+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:57.222996+0000 front 2024-03-24T11:34:57.223035+0000 (oldest deadline 2024-03-24T11:35:20.122977+0000) 2024-03-24T11:35:33.858 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:33 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:33.556+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6822 osd.2 since back 2024-03-24T11:35:06.024102+0000 front 2024-03-24T11:35:06.024093+0000 (oldest deadline 2024-03-24T11:35:31.323890+0000) 2024-03-24T11:35:34.358 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:35:34 smithi139 podman[156605]: 2024-03-24 11:35:33.970451944 +0000 UTC m=+0.651787100 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:35:34.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:34 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:34.069+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.556704+0000 front 2024-03-24T11:34:47.556794+0000 (oldest deadline 2024-03-24T11:35:12.856557+0000) 2024-03-24T11:35:34.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:34 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:34.069+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:58.657782+0000 front 2024-03-24T11:34:58.657834+0000 (oldest deadline 2024-03-24T11:35:20.357736+0000) 2024-03-24T11:35:34.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:34 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:34.069+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6822 osd.2 since back 2024-03-24T11:35:06.258940+0000 front 2024-03-24T11:35:06.258952+0000 (oldest deadline 2024-03-24T11:35:29.758769+0000) 2024-03-24T11:35:34.708 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:35:34 smithi139 podman[156605]: 2024-03-24 11:35:34.601479807 +0000 UTC m=+1.282815004 container create 11ba66a859a427c076c33ad08bfcd9f3f0034a26933e17bae0b12251c5d72a18 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True) 2024-03-24T11:35:34.708 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:34 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:34.561+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.222394+0000 front 2024-03-24T11:34:47.222378+0000 (oldest deadline 2024-03-24T11:35:13.122152+0000) 2024-03-24T11:35:34.708 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:34 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:34.561+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:57.222996+0000 front 2024-03-24T11:34:57.223035+0000 (oldest deadline 2024-03-24T11:35:20.122977+0000) 2024-03-24T11:35:34.708 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:34 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:34.561+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6822 osd.2 since back 2024-03-24T11:35:06.024102+0000 front 2024-03-24T11:35:06.024093+0000 (oldest deadline 2024-03-24T11:35:31.323890+0000) 2024-03-24T11:35:35.284 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:35 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:35.030+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.556704+0000 front 2024-03-24T11:34:47.556794+0000 (oldest deadline 2024-03-24T11:35:12.856557+0000) 2024-03-24T11:35:35.284 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:35 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:35.030+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:58.657782+0000 front 2024-03-24T11:34:58.657834+0000 (oldest deadline 2024-03-24T11:35:20.357736+0000) 2024-03-24T11:35:35.284 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:35 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:35.030+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6822 osd.2 since back 2024-03-24T11:35:06.258940+0000 front 2024-03-24T11:35:06.258952+0000 (oldest deadline 2024-03-24T11:35:29.758769+0000) 2024-03-24T11:35:35.827 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:35:35 smithi139 podman[156605]: 2024-03-24 11:35:35.540470637 +0000 UTC m=+2.221805793 container init 11ba66a859a427c076c33ad08bfcd9f3f0034a26933e17bae0b12251c5d72a18 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-db0330b, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307) 2024-03-24T11:35:35.827 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:35:35 smithi139 podman[156605]: 2024-03-24 11:35:35.543719478 +0000 UTC m=+2.225054634 container start 11ba66a859a427c076c33ad08bfcd9f3f0034a26933e17bae0b12251c5d72a18 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-24T11:35:35.828 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:35:35 smithi139 podman[156605]: 2024-03-24 11:35:35.640635471 +0000 UTC m=+2.321970636 container attach 11ba66a859a427c076c33ad08bfcd9f3f0034a26933e17bae0b12251c5d72a18 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=squid-db0330b, GIT_CLEAN=True, ceph=True) 2024-03-24T11:35:35.828 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:35 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:35.606+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.222394+0000 front 2024-03-24T11:34:47.222378+0000 (oldest deadline 2024-03-24T11:35:13.122152+0000) 2024-03-24T11:35:35.828 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:35 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:35.606+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:57.222996+0000 front 2024-03-24T11:34:57.223035+0000 (oldest deadline 2024-03-24T11:35:20.122977+0000) 2024-03-24T11:35:35.828 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:35 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:35.606+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6822 osd.2 since back 2024-03-24T11:35:06.024102+0000 front 2024-03-24T11:35:06.024093+0000 (oldest deadline 2024-03-24T11:35:31.323890+0000) 2024-03-24T11:35:35.828 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:35 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:35.606+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6830 osd.3 since back 2024-03-24T11:35:11.324480+0000 front 2024-03-24T11:35:11.324413+0000 (oldest deadline 2024-03-24T11:35:35.424291+0000) 2024-03-24T11:35:36.108 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:35 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:35.996+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.556704+0000 front 2024-03-24T11:34:47.556794+0000 (oldest deadline 2024-03-24T11:35:12.856557+0000) 2024-03-24T11:35:36.108 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:35 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:35.996+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:58.657782+0000 front 2024-03-24T11:34:58.657834+0000 (oldest deadline 2024-03-24T11:35:20.357736+0000) 2024-03-24T11:35:36.108 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:35 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:35.996+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6822 osd.2 since back 2024-03-24T11:35:06.258940+0000 front 2024-03-24T11:35:06.258952+0000 (oldest deadline 2024-03-24T11:35:29.758769+0000) 2024-03-24T11:35:36.108 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:35 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:35.996+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6830 osd.3 since back 2024-03-24T11:35:10.859841+0000 front 2024-03-24T11:35:10.859784+0000 (oldest deadline 2024-03-24T11:35:35.559676+0000) 2024-03-24T11:35:36.595 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:35:36 smithi139 podman[156605]: 2024-03-24 11:35:36.277810559 +0000 UTC m=+2.959145724 container died 11ba66a859a427c076c33ad08bfcd9f3f0034a26933e17bae0b12251c5d72a18 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-db0330b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-03-24T11:35:36.848 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:36 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:36.594+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.222394+0000 front 2024-03-24T11:34:47.222378+0000 (oldest deadline 2024-03-24T11:35:13.122152+0000) 2024-03-24T11:35:36.848 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:36 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:36.594+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:57.222996+0000 front 2024-03-24T11:34:57.223035+0000 (oldest deadline 2024-03-24T11:35:20.122977+0000) 2024-03-24T11:35:36.848 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:36 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:36.594+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6822 osd.2 since back 2024-03-24T11:35:06.024102+0000 front 2024-03-24T11:35:06.024093+0000 (oldest deadline 2024-03-24T11:35:31.323890+0000) 2024-03-24T11:35:36.848 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:36 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:36.594+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6830 osd.3 since back 2024-03-24T11:35:11.324480+0000 front 2024-03-24T11:35:11.324413+0000 (oldest deadline 2024-03-24T11:35:35.424291+0000) 2024-03-24T11:35:37.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:37 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:37.003+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.556704+0000 front 2024-03-24T11:34:47.556794+0000 (oldest deadline 2024-03-24T11:35:12.856557+0000) 2024-03-24T11:35:37.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:37 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:37.003+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:58.657782+0000 front 2024-03-24T11:34:58.657834+0000 (oldest deadline 2024-03-24T11:35:20.357736+0000) 2024-03-24T11:35:37.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:37 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:37.003+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6822 osd.2 since back 2024-03-24T11:35:06.258940+0000 front 2024-03-24T11:35:06.258952+0000 (oldest deadline 2024-03-24T11:35:29.758769+0000) 2024-03-24T11:35:37.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:37 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:37.003+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6830 osd.3 since back 2024-03-24T11:35:10.859841+0000 front 2024-03-24T11:35:10.859784+0000 (oldest deadline 2024-03-24T11:35:35.559676+0000) 2024-03-24T11:35:37.619 INFO:journalctl@ceph.osd.5.smithi139.stdout:Mar 24 11:35:37 smithi139 podman[156819]: 2024-03-24 11:35:37.601784902 +0000 UTC m=+1.748017732 container remove 11ba66a859a427c076c33ad08bfcd9f3f0034a26933e17bae0b12251c5d72a18 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-5-deactivate, RELEASE=squid-db0330b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-24T11:35:37.626 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.5.service' 2024-03-24T11:35:37.668 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T11:35:37.668 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-24T11:35:37.668 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-03-24T11:35:37.668 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.6 2024-03-24T11:35:37.921 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:37 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:37.624+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.222394+0000 front 2024-03-24T11:34:47.222378+0000 (oldest deadline 2024-03-24T11:35:13.122152+0000) 2024-03-24T11:35:37.922 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:37 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:37.624+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:57.222996+0000 front 2024-03-24T11:34:57.223035+0000 (oldest deadline 2024-03-24T11:35:20.122977+0000) 2024-03-24T11:35:37.922 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:37 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:37.624+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6822 osd.2 since back 2024-03-24T11:35:06.024102+0000 front 2024-03-24T11:35:06.024093+0000 (oldest deadline 2024-03-24T11:35:31.323890+0000) 2024-03-24T11:35:37.922 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:37 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:37.624+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6830 osd.3 since back 2024-03-24T11:35:11.324480+0000 front 2024-03-24T11:35:11.324413+0000 (oldest deadline 2024-03-24T11:35:35.424291+0000) 2024-03-24T11:35:37.922 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:37 smithi139 systemd[1]: Stopping Ceph osd.6 for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:35:38.305 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:38 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:38.024+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.556704+0000 front 2024-03-24T11:34:47.556794+0000 (oldest deadline 2024-03-24T11:35:12.856557+0000) 2024-03-24T11:35:38.305 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:38 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:38.024+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:58.657782+0000 front 2024-03-24T11:34:58.657834+0000 (oldest deadline 2024-03-24T11:35:20.357736+0000) 2024-03-24T11:35:38.305 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:38 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:38.024+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6822 osd.2 since back 2024-03-24T11:35:06.258940+0000 front 2024-03-24T11:35:06.258952+0000 (oldest deadline 2024-03-24T11:35:29.758769+0000) 2024-03-24T11:35:38.305 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:38 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:38.024+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6830 osd.3 since back 2024-03-24T11:35:10.859841+0000 front 2024-03-24T11:35:10.859784+0000 (oldest deadline 2024-03-24T11:35:35.559676+0000) 2024-03-24T11:35:38.981 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:38 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:38.615+0000 7fa785236700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T11:35:38.981 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:38 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:38.615+0000 7fa785236700 -1 osd.6 180 *** Got signal Terminated *** 2024-03-24T11:35:38.981 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:38 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:38.615+0000 7fa785236700 -1 osd.6 180 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T11:35:38.981 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:38 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:38.630+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.222394+0000 front 2024-03-24T11:34:47.222378+0000 (oldest deadline 2024-03-24T11:35:13.122152+0000) 2024-03-24T11:35:38.981 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:38 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:38.630+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:57.222996+0000 front 2024-03-24T11:34:57.223035+0000 (oldest deadline 2024-03-24T11:35:20.122977+0000) 2024-03-24T11:35:38.981 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:38 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:38.630+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6822 osd.2 since back 2024-03-24T11:35:06.024102+0000 front 2024-03-24T11:35:06.024093+0000 (oldest deadline 2024-03-24T11:35:31.323890+0000) 2024-03-24T11:35:38.981 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:38 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:38.630+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6830 osd.3 since back 2024-03-24T11:35:11.324480+0000 front 2024-03-24T11:35:11.324413+0000 (oldest deadline 2024-03-24T11:35:35.424291+0000) 2024-03-24T11:35:39.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:38 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:38.979+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.556704+0000 front 2024-03-24T11:34:47.556794+0000 (oldest deadline 2024-03-24T11:35:12.856557+0000) 2024-03-24T11:35:39.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:38 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:38.979+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:58.657782+0000 front 2024-03-24T11:34:58.657834+0000 (oldest deadline 2024-03-24T11:35:20.357736+0000) 2024-03-24T11:35:39.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:38 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:38.979+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6822 osd.2 since back 2024-03-24T11:35:06.258940+0000 front 2024-03-24T11:35:06.258952+0000 (oldest deadline 2024-03-24T11:35:29.758769+0000) 2024-03-24T11:35:39.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:38 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:38.979+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6830 osd.3 since back 2024-03-24T11:35:10.859841+0000 front 2024-03-24T11:35:10.859784+0000 (oldest deadline 2024-03-24T11:35:35.559676+0000) 2024-03-24T11:35:39.932 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:39 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:39.663+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.222394+0000 front 2024-03-24T11:34:47.222378+0000 (oldest deadline 2024-03-24T11:35:13.122152+0000) 2024-03-24T11:35:39.932 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:39 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:39.663+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:57.222996+0000 front 2024-03-24T11:34:57.223035+0000 (oldest deadline 2024-03-24T11:35:20.122977+0000) 2024-03-24T11:35:39.933 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:39 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:39.663+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6822 osd.2 since back 2024-03-24T11:35:06.024102+0000 front 2024-03-24T11:35:06.024093+0000 (oldest deadline 2024-03-24T11:35:31.323890+0000) 2024-03-24T11:35:39.933 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:39 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:39.663+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6830 osd.3 since back 2024-03-24T11:35:11.324480+0000 front 2024-03-24T11:35:11.324413+0000 (oldest deadline 2024-03-24T11:35:35.424291+0000) 2024-03-24T11:35:40.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:39 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:39.931+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.556704+0000 front 2024-03-24T11:34:47.556794+0000 (oldest deadline 2024-03-24T11:35:12.856557+0000) 2024-03-24T11:35:40.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:39 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:39.931+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:58.657782+0000 front 2024-03-24T11:34:58.657834+0000 (oldest deadline 2024-03-24T11:35:20.357736+0000) 2024-03-24T11:35:40.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:39 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:39.931+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6822 osd.2 since back 2024-03-24T11:35:06.258940+0000 front 2024-03-24T11:35:06.258952+0000 (oldest deadline 2024-03-24T11:35:29.758769+0000) 2024-03-24T11:35:40.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:39 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:39.931+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6830 osd.3 since back 2024-03-24T11:35:10.859841+0000 front 2024-03-24T11:35:10.859784+0000 (oldest deadline 2024-03-24T11:35:35.559676+0000) 2024-03-24T11:35:40.975 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:40 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:40.643+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.222394+0000 front 2024-03-24T11:34:47.222378+0000 (oldest deadline 2024-03-24T11:35:13.122152+0000) 2024-03-24T11:35:40.975 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:40 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:40.643+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:57.222996+0000 front 2024-03-24T11:34:57.223035+0000 (oldest deadline 2024-03-24T11:35:20.122977+0000) 2024-03-24T11:35:40.975 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:40 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:40.643+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6822 osd.2 since back 2024-03-24T11:35:06.024102+0000 front 2024-03-24T11:35:06.024093+0000 (oldest deadline 2024-03-24T11:35:31.323890+0000) 2024-03-24T11:35:40.975 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:40 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:40.643+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6830 osd.3 since back 2024-03-24T11:35:11.324480+0000 front 2024-03-24T11:35:11.324413+0000 (oldest deadline 2024-03-24T11:35:35.424291+0000) 2024-03-24T11:35:41.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:40 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:40.974+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.556704+0000 front 2024-03-24T11:34:47.556794+0000 (oldest deadline 2024-03-24T11:35:12.856557+0000) 2024-03-24T11:35:41.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:40 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:40.974+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:58.657782+0000 front 2024-03-24T11:34:58.657834+0000 (oldest deadline 2024-03-24T11:35:20.357736+0000) 2024-03-24T11:35:41.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:40 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:40.974+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6822 osd.2 since back 2024-03-24T11:35:06.258940+0000 front 2024-03-24T11:35:06.258952+0000 (oldest deadline 2024-03-24T11:35:29.758769+0000) 2024-03-24T11:35:41.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:40 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:40.974+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6830 osd.3 since back 2024-03-24T11:35:10.859841+0000 front 2024-03-24T11:35:10.859784+0000 (oldest deadline 2024-03-24T11:35:35.559676+0000) 2024-03-24T11:35:41.954 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:41 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:41.658+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.222394+0000 front 2024-03-24T11:34:47.222378+0000 (oldest deadline 2024-03-24T11:35:13.122152+0000) 2024-03-24T11:35:41.954 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:41 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:41.658+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:57.222996+0000 front 2024-03-24T11:34:57.223035+0000 (oldest deadline 2024-03-24T11:35:20.122977+0000) 2024-03-24T11:35:41.955 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:41 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:41.658+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6822 osd.2 since back 2024-03-24T11:35:06.024102+0000 front 2024-03-24T11:35:06.024093+0000 (oldest deadline 2024-03-24T11:35:31.323890+0000) 2024-03-24T11:35:41.955 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:41 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:41.658+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6830 osd.3 since back 2024-03-24T11:35:11.324480+0000 front 2024-03-24T11:35:11.324413+0000 (oldest deadline 2024-03-24T11:35:35.424291+0000) 2024-03-24T11:35:42.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:41 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:41.953+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.556704+0000 front 2024-03-24T11:34:47.556794+0000 (oldest deadline 2024-03-24T11:35:12.856557+0000) 2024-03-24T11:35:42.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:41 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:41.953+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:58.657782+0000 front 2024-03-24T11:34:58.657834+0000 (oldest deadline 2024-03-24T11:35:20.357736+0000) 2024-03-24T11:35:42.359 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:41 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:41.953+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6822 osd.2 since back 2024-03-24T11:35:06.258940+0000 front 2024-03-24T11:35:06.258952+0000 (oldest deadline 2024-03-24T11:35:29.758769+0000) 2024-03-24T11:35:42.359 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:41 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:41.953+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6830 osd.3 since back 2024-03-24T11:35:10.859841+0000 front 2024-03-24T11:35:10.859784+0000 (oldest deadline 2024-03-24T11:35:35.559676+0000) 2024-03-24T11:35:43.004 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:42 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:42.673+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.222394+0000 front 2024-03-24T11:34:47.222378+0000 (oldest deadline 2024-03-24T11:35:13.122152+0000) 2024-03-24T11:35:43.004 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:42 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:42.673+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:57.222996+0000 front 2024-03-24T11:34:57.223035+0000 (oldest deadline 2024-03-24T11:35:20.122977+0000) 2024-03-24T11:35:43.004 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:42 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:42.673+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6822 osd.2 since back 2024-03-24T11:35:06.024102+0000 front 2024-03-24T11:35:06.024093+0000 (oldest deadline 2024-03-24T11:35:31.323890+0000) 2024-03-24T11:35:43.004 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:42 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6[136654]: 2024-03-24T11:35:42.673+0000 7fa78182a700 -1 osd.6 180 heartbeat_check: no reply from 172.21.15.62:6830 osd.3 since back 2024-03-24T11:35:11.324480+0000 front 2024-03-24T11:35:11.324413+0000 (oldest deadline 2024-03-24T11:35:35.424291+0000) 2024-03-24T11:35:43.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:43 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:43.003+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.556704+0000 front 2024-03-24T11:34:47.556794+0000 (oldest deadline 2024-03-24T11:35:12.856557+0000) 2024-03-24T11:35:43.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:43 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:43.003+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:58.657782+0000 front 2024-03-24T11:34:58.657834+0000 (oldest deadline 2024-03-24T11:35:20.357736+0000) 2024-03-24T11:35:43.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:43 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:43.003+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6822 osd.2 since back 2024-03-24T11:35:06.258940+0000 front 2024-03-24T11:35:06.258952+0000 (oldest deadline 2024-03-24T11:35:29.758769+0000) 2024-03-24T11:35:43.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:43 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:43.003+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6830 osd.3 since back 2024-03-24T11:35:10.859841+0000 front 2024-03-24T11:35:10.859784+0000 (oldest deadline 2024-03-24T11:35:35.559676+0000) 2024-03-24T11:35:43.836 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:35:43 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[64072]: level=error ts=2024-03-24T11:35:43.558Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=6 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi139.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.139:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi062.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.62:8443: connect: connection refused" 2024-03-24T11:35:43.836 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:35:43 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[64072]: level=warn ts=2024-03-24T11:35:43.559Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi062.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.62:8443: connect: connection refused" 2024-03-24T11:35:43.836 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:35:43 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[64072]: level=warn ts=2024-03-24T11:35:43.560Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi139.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.139:8443: connect: connection refused" 2024-03-24T11:35:44.326 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:44 smithi139 podman[157003]: 2024-03-24 11:35:44.05291646 +0000 UTC m=+5.530147411 container died c67fd9426ac5cb96eb7389e2f66c5f38f3e9c518dd9aad1e68a6a8520bc41d7d (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-db0330b, io.buildah.version=1.33.5) 2024-03-24T11:35:44.326 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:44 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:44.011+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.556704+0000 front 2024-03-24T11:34:47.556794+0000 (oldest deadline 2024-03-24T11:35:12.856557+0000) 2024-03-24T11:35:44.326 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:44 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:44.011+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:58.657782+0000 front 2024-03-24T11:34:58.657834+0000 (oldest deadline 2024-03-24T11:35:20.357736+0000) 2024-03-24T11:35:44.326 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:44 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:44.011+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6822 osd.2 since back 2024-03-24T11:35:06.258940+0000 front 2024-03-24T11:35:06.258952+0000 (oldest deadline 2024-03-24T11:35:29.758769+0000) 2024-03-24T11:35:44.326 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:44 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:44.011+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6830 osd.3 since back 2024-03-24T11:35:10.859841+0000 front 2024-03-24T11:35:10.859784+0000 (oldest deadline 2024-03-24T11:35:35.559676+0000) 2024-03-24T11:35:44.608 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:44 smithi139 podman[157003]: 2024-03-24 11:35:44.474421234 +0000 UTC m=+5.951652185 container cleanup c67fd9426ac5cb96eb7389e2f66c5f38f3e9c518dd9aad1e68a6a8520bc41d7d (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-db0330b, ceph=True) 2024-03-24T11:35:44.608 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:44 smithi139 bash[157003]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6 2024-03-24T11:35:45.009 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:44 smithi139 podman[157022]: 2024-03-24 11:35:44.701267397 +0000 UTC m=+0.644588549 container remove c67fd9426ac5cb96eb7389e2f66c5f38f3e9c518dd9aad1e68a6a8520bc41d7d (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T11:35:45.264 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:45 smithi139 podman[157113]: 2024-03-24 11:35:45.135294504 +0000 UTC m=+0.110064585 container create 523792fd312617a66d96535491f5d8fb14783200ae57ce97c5d19d1aee65771b (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=squid-db0330b, GIT_CLEAN=True, ceph=True) 2024-03-24T11:35:45.264 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:45 smithi139 podman[157113]: 2024-03-24 11:35:45.04321361 +0000 UTC m=+0.017983719 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:35:45.264 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:45 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:45.008+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.556704+0000 front 2024-03-24T11:34:47.556794+0000 (oldest deadline 2024-03-24T11:35:12.856557+0000) 2024-03-24T11:35:45.264 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:45 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:45.008+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:58.657782+0000 front 2024-03-24T11:34:58.657834+0000 (oldest deadline 2024-03-24T11:35:20.357736+0000) 2024-03-24T11:35:45.264 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:45 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:45.008+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6822 osd.2 since back 2024-03-24T11:35:06.258940+0000 front 2024-03-24T11:35:06.258952+0000 (oldest deadline 2024-03-24T11:35:29.758769+0000) 2024-03-24T11:35:45.264 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:45 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:45.008+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6830 osd.3 since back 2024-03-24T11:35:10.859841+0000 front 2024-03-24T11:35:10.859784+0000 (oldest deadline 2024-03-24T11:35:35.559676+0000) 2024-03-24T11:35:45.608 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:45 smithi139 podman[157113]: 2024-03-24 11:35:45.346161577 +0000 UTC m=+0.320931676 container init 523792fd312617a66d96535491f5d8fb14783200ae57ce97c5d19d1aee65771b (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, RELEASE=squid-db0330b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-24T11:35:45.608 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:45 smithi139 podman[157113]: 2024-03-24 11:35:45.349244424 +0000 UTC m=+0.324014515 container start 523792fd312617a66d96535491f5d8fb14783200ae57ce97c5d19d1aee65771b (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-24T11:35:45.608 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:45 smithi139 podman[157113]: 2024-03-24 11:35:45.429199708 +0000 UTC m=+0.403969793 container attach 523792fd312617a66d96535491f5d8fb14783200ae57ce97c5d19d1aee65771b (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-db0330b, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T11:35:45.924 INFO:journalctl@ceph.osd.6.smithi139.stdout:Mar 24 11:35:45 smithi139 podman[157113]: 2024-03-24 11:35:45.637586852 +0000 UTC m=+0.612356925 container died 523792fd312617a66d96535491f5d8fb14783200ae57ce97c5d19d1aee65771b (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-6-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-db0330b, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD) 2024-03-24T11:35:46.182 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:46 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:46.037+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.556704+0000 front 2024-03-24T11:34:47.556794+0000 (oldest deadline 2024-03-24T11:35:12.856557+0000) 2024-03-24T11:35:46.182 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:46 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:46.037+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:58.657782+0000 front 2024-03-24T11:34:58.657834+0000 (oldest deadline 2024-03-24T11:35:20.357736+0000) 2024-03-24T11:35:46.182 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:46 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:46.037+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6822 osd.2 since back 2024-03-24T11:35:06.258940+0000 front 2024-03-24T11:35:06.258952+0000 (oldest deadline 2024-03-24T11:35:29.758769+0000) 2024-03-24T11:35:46.183 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:46 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:46.037+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6830 osd.3 since back 2024-03-24T11:35:10.859841+0000 front 2024-03-24T11:35:10.859784+0000 (oldest deadline 2024-03-24T11:35:35.559676+0000) 2024-03-24T11:35:46.212 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.6.service' 2024-03-24T11:35:46.252 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T11:35:46.252 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-03-24T11:35:46.253 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-03-24T11:35:46.253 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.7 2024-03-24T11:35:46.568 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:46 smithi139 systemd[1]: Stopping Ceph osd.7 for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:35:46.858 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:46 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:46.567+0000 7fe410017700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T11:35:46.858 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:46 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:46.567+0000 7fe410017700 -1 osd.7 180 *** Got signal Terminated *** 2024-03-24T11:35:46.858 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:46 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:46.567+0000 7fe410017700 -1 osd.7 180 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T11:35:47.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:47.051+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.556704+0000 front 2024-03-24T11:34:47.556794+0000 (oldest deadline 2024-03-24T11:35:12.856557+0000) 2024-03-24T11:35:47.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:47.051+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:58.657782+0000 front 2024-03-24T11:34:58.657834+0000 (oldest deadline 2024-03-24T11:35:20.357736+0000) 2024-03-24T11:35:47.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:47.051+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6822 osd.2 since back 2024-03-24T11:35:06.258940+0000 front 2024-03-24T11:35:06.258952+0000 (oldest deadline 2024-03-24T11:35:29.758769+0000) 2024-03-24T11:35:47.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:47.051+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6830 osd.3 since back 2024-03-24T11:35:10.859841+0000 front 2024-03-24T11:35:10.859784+0000 (oldest deadline 2024-03-24T11:35:35.559676+0000) 2024-03-24T11:35:47.359 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:47 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:47.051+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.139:6804 osd.4 since back 2024-03-24T11:35:22.361939+0000 front 2024-03-24T11:35:22.361908+0000 (oldest deadline 2024-03-24T11:35:46.461878+0000) 2024-03-24T11:35:48.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:48.061+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.556704+0000 front 2024-03-24T11:34:47.556794+0000 (oldest deadline 2024-03-24T11:35:12.856557+0000) 2024-03-24T11:35:48.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:48.061+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:58.657782+0000 front 2024-03-24T11:34:58.657834+0000 (oldest deadline 2024-03-24T11:35:20.357736+0000) 2024-03-24T11:35:48.359 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:48.061+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6822 osd.2 since back 2024-03-24T11:35:06.258940+0000 front 2024-03-24T11:35:06.258952+0000 (oldest deadline 2024-03-24T11:35:29.758769+0000) 2024-03-24T11:35:48.359 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:48.061+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6830 osd.3 since back 2024-03-24T11:35:10.859841+0000 front 2024-03-24T11:35:10.859784+0000 (oldest deadline 2024-03-24T11:35:35.559676+0000) 2024-03-24T11:35:48.359 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:48.061+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.139:6804 osd.4 since back 2024-03-24T11:35:22.361939+0000 front 2024-03-24T11:35:22.361908+0000 (oldest deadline 2024-03-24T11:35:46.461878+0000) 2024-03-24T11:35:48.359 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:48 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:48.061+0000 7fe40c60b700 -1 osd.7 180 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.44766.0:39 2.0 2:06e376a1:::realms.937f0dc2-2b1d-4d64-b23a-bf9f2400aa57.control:head [watch ping cookie 94910613780096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e180) 2024-03-24T11:35:49.608 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:49 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:49.111+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.556704+0000 front 2024-03-24T11:34:47.556794+0000 (oldest deadline 2024-03-24T11:35:12.856557+0000) 2024-03-24T11:35:49.608 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:49 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:49.111+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:58.657782+0000 front 2024-03-24T11:34:58.657834+0000 (oldest deadline 2024-03-24T11:35:20.357736+0000) 2024-03-24T11:35:49.608 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:49 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:49.111+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6822 osd.2 since back 2024-03-24T11:35:06.258940+0000 front 2024-03-24T11:35:06.258952+0000 (oldest deadline 2024-03-24T11:35:29.758769+0000) 2024-03-24T11:35:49.608 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:49 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:49.111+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6830 osd.3 since back 2024-03-24T11:35:10.859841+0000 front 2024-03-24T11:35:10.859784+0000 (oldest deadline 2024-03-24T11:35:35.559676+0000) 2024-03-24T11:35:49.608 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:49 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:49.111+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.139:6804 osd.4 since back 2024-03-24T11:35:22.361939+0000 front 2024-03-24T11:35:22.361908+0000 (oldest deadline 2024-03-24T11:35:46.461878+0000) 2024-03-24T11:35:49.608 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:49 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:49.111+0000 7fe40c60b700 -1 osd.7 180 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.44766.0:39 2.0 2:06e376a1:::realms.937f0dc2-2b1d-4d64-b23a-bf9f2400aa57.control:head [watch ping cookie 94910613780096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e180) 2024-03-24T11:35:50.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:50 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:50.094+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.556704+0000 front 2024-03-24T11:34:47.556794+0000 (oldest deadline 2024-03-24T11:35:12.856557+0000) 2024-03-24T11:35:50.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:50 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:50.094+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:58.657782+0000 front 2024-03-24T11:34:58.657834+0000 (oldest deadline 2024-03-24T11:35:20.357736+0000) 2024-03-24T11:35:50.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:50 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:50.094+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6822 osd.2 since back 2024-03-24T11:35:06.258940+0000 front 2024-03-24T11:35:06.258952+0000 (oldest deadline 2024-03-24T11:35:29.758769+0000) 2024-03-24T11:35:50.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:50 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:50.094+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6830 osd.3 since back 2024-03-24T11:35:10.859841+0000 front 2024-03-24T11:35:10.859784+0000 (oldest deadline 2024-03-24T11:35:35.559676+0000) 2024-03-24T11:35:50.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:50 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:50.094+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.139:6804 osd.4 since back 2024-03-24T11:35:22.361939+0000 front 2024-03-24T11:35:22.361908+0000 (oldest deadline 2024-03-24T11:35:46.461878+0000) 2024-03-24T11:35:50.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:50 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:50.094+0000 7fe40c60b700 -1 osd.7 180 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.44766.0:39 2.0 2:06e376a1:::realms.937f0dc2-2b1d-4d64-b23a-bf9f2400aa57.control:head [watch ping cookie 94910613780096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e180) 2024-03-24T11:35:51.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:51 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:51.071+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-03-24T11:34:47.556704+0000 front 2024-03-24T11:34:47.556794+0000 (oldest deadline 2024-03-24T11:35:12.856557+0000) 2024-03-24T11:35:51.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:51 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:51.072+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6814 osd.1 since back 2024-03-24T11:34:58.657782+0000 front 2024-03-24T11:34:58.657834+0000 (oldest deadline 2024-03-24T11:35:20.357736+0000) 2024-03-24T11:35:51.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:51 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:51.072+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6822 osd.2 since back 2024-03-24T11:35:06.258940+0000 front 2024-03-24T11:35:06.258952+0000 (oldest deadline 2024-03-24T11:35:29.758769+0000) 2024-03-24T11:35:51.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:51 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:51.072+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.62:6830 osd.3 since back 2024-03-24T11:35:10.859841+0000 front 2024-03-24T11:35:10.859784+0000 (oldest deadline 2024-03-24T11:35:35.559676+0000) 2024-03-24T11:35:51.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:51 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:51.072+0000 7fe40c60b700 -1 osd.7 180 heartbeat_check: no reply from 172.21.15.139:6804 osd.4 since back 2024-03-24T11:35:22.361939+0000 front 2024-03-24T11:35:22.361908+0000 (oldest deadline 2024-03-24T11:35:46.461878+0000) 2024-03-24T11:35:51.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:51 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7[138855]: 2024-03-24T11:35:51.072+0000 7fe40c60b700 -1 osd.7 180 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.44766.0:39 2.0 2:06e376a1:::realms.937f0dc2-2b1d-4d64-b23a-bf9f2400aa57.control:head [watch ping cookie 94910613780096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e180) 2024-03-24T11:35:52.352 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:52 smithi139 podman[157220]: 2024-03-24 11:35:52.055048466 +0000 UTC m=+5.580772768 container died d4fbd9bccd09fdf1dbef693bb02c64a92d96f60b75188faf96c0edef5a4f4cf3 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=squid-db0330b) 2024-03-24T11:35:52.608 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:52 smithi139 podman[157220]: 2024-03-24 11:35:52.526755511 +0000 UTC m=+6.052479796 container cleanup d4fbd9bccd09fdf1dbef693bb02c64a92d96f60b75188faf96c0edef5a4f4cf3 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-db0330b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-24T11:35:52.608 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:52 smithi139 bash[157220]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7 2024-03-24T11:35:53.087 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:52 smithi139 podman[157237]: 2024-03-24 11:35:52.769818394 +0000 UTC m=+0.706853063 container remove d4fbd9bccd09fdf1dbef693bb02c64a92d96f60b75188faf96c0edef5a4f4cf3 (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-db0330b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-24T11:35:53.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:53 smithi139 podman[157330]: 2024-03-24 11:35:52.989768416 +0000 UTC m=+0.017468257 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:35:53.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:53 smithi139 podman[157330]: 2024-03-24 11:35:53.095338073 +0000 UTC m=+0.123037896 container create 9f12d0e5877a06766e97c81c31f17ecf49832812f47e1ea7f5ab40d4847d330c (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7-deactivate, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-24T11:35:53.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:53 smithi139 podman[157330]: 2024-03-24 11:35:53.306124757 +0000 UTC m=+0.333824588 container init 9f12d0e5877a06766e97c81c31f17ecf49832812f47e1ea7f5ab40d4847d330c (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, RELEASE=squid-db0330b, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T11:35:53.358 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:53 smithi139 podman[157330]: 2024-03-24 11:35:53.309226988 +0000 UTC m=+0.336926814 container start 9f12d0e5877a06766e97c81c31f17ecf49832812f47e1ea7f5ab40d4847d330c (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, RELEASE=squid-db0330b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-24T11:35:53.650 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:53 smithi139 podman[157330]: 2024-03-24 11:35:53.391655529 +0000 UTC m=+0.419355369 container attach 9f12d0e5877a06766e97c81c31f17ecf49832812f47e1ea7f5ab40d4847d330c (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-db0330b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-24T11:35:53.650 INFO:journalctl@ceph.osd.7.smithi139.stdout:Mar 24 11:35:53 smithi139 podman[157330]: 2024-03-24 11:35:53.632092852 +0000 UTC m=+0.659792676 container died 9f12d0e5877a06766e97c81c31f17ecf49832812f47e1ea7f5ab40d4847d330c (image=quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-osd-7-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-db0330b, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T11:35:53.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:35:53 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[64072]: level=error ts=2024-03-24T11:35:53.559Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=6 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi062.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.62:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi139.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.139:8443: connect: connection refused" 2024-03-24T11:35:53.978 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:35:53 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[64072]: level=warn ts=2024-03-24T11:35:53.560Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi062.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.62:8443: connect: connection refused" 2024-03-24T11:35:53.978 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:35:53 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[64072]: level=warn ts=2024-03-24T11:35:53.560Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi139.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.139:8443: connect: connection refused" 2024-03-24T11:35:54.332 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@osd.7.service' 2024-03-24T11:35:54.374 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T11:35:54.375 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-03-24T11:35:54.375 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-03-24T11:35:54.375 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@prometheus.a 2024-03-24T11:35:54.579 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:35:54 smithi139 systemd[1]: Stopping Ceph prometheus.a for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:35:54.858 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:35:54 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:35:54.579Z caller=main.go:775 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-03-24T11:35:54.858 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:35:54 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:35:54.579Z caller=main.go:798 level=info msg="Stopping scrape discovery manager..." 2024-03-24T11:35:54.858 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:35:54 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:35:54.579Z caller=main.go:812 level=info msg="Stopping notify discovery manager..." 2024-03-24T11:35:54.859 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:35:54 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:35:54.579Z caller=main.go:834 level=info msg="Stopping scrape manager..." 2024-03-24T11:35:54.859 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:35:54 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:35:54.579Z caller=main.go:794 level=info msg="Scrape discovery manager stopped" 2024-03-24T11:35:54.859 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:35:54 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:35:54.579Z caller=main.go:808 level=info msg="Notify discovery manager stopped" 2024-03-24T11:35:54.859 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:35:54 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:35:54.579Z caller=manager.go:945 level=info component="rule manager" msg="Stopping rule manager..." 2024-03-24T11:35:54.859 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:35:54 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:35:54.579Z caller=main.go:828 level=info msg="Scrape manager stopped" 2024-03-24T11:35:54.859 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:35:54 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:35:54.579Z caller=manager.go:955 level=info component="rule manager" msg="Rule manager stopped" 2024-03-24T11:35:54.859 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:35:54 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:35:54.580Z caller=notifier.go:600 level=info component=notifier msg="Stopping notification manager..." 2024-03-24T11:35:54.859 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:35:54 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:35:54.580Z caller=main.go:1054 level=info msg="Notifier manager stopped" 2024-03-24T11:35:54.859 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:35:54 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a[60700]: ts=2024-03-24T11:35:54.580Z caller=main.go:1066 level=info msg="See you next time!" 2024-03-24T11:35:54.859 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:35:54 smithi139 podman[157544]: 2024-03-24 11:35:54.589864788 +0000 UTC m=+0.122788328 container died 2ed47f810d33a3b1e18ee293bfcbe12275d0c43227d2948938e31d807355fcb7 (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-24T11:35:55.204 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:35:54 smithi139 podman[157544]: 2024-03-24 11:35:54.953865627 +0000 UTC m=+0.486789160 container cleanup 2ed47f810d33a3b1e18ee293bfcbe12275d0c43227d2948938e31d807355fcb7 (image=quay.io/prometheus/prometheus:v2.33.4, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-24T11:35:55.204 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:35:54 smithi139 bash[157544]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus-a 2024-03-24T11:35:55.205 INFO:journalctl@ceph.prometheus.a.smithi139.stdout:Mar 24 11:35:54 smithi139 bash[157589]: Error: no container with name or ID "ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-prometheus.a" found: no such container 2024-03-24T11:35:55.209 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@prometheus.a.service' 2024-03-24T11:35:55.249 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T11:35:55.249 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-03-24T11:35:55.250 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-24T11:35:57.045 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:35:56 smithi062 systemd[1]: Stopping Ceph node-exporter.a for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:35:57.301 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:35:57 smithi062 podman[183291]: 2024-03-24 11:35:57.04472738 +0000 UTC m=+0.111131055 container died 79d5a02a999fc15cd0f03e2d8e2a652d6c9b8027e1077f90df4608a1ce5ec9d8 (image=quay.io/prometheus/node-exporter:v1.3.1, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-03-24T11:35:57.643 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:35:57 smithi062 podman[183291]: 2024-03-24 11:35:57.392737355 +0000 UTC m=+0.459141038 container cleanup 79d5a02a999fc15cd0f03e2d8e2a652d6c9b8027e1077f90df4608a1ce5ec9d8 (image=quay.io/prometheus/node-exporter:v1.3.1, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-03-24T11:35:57.643 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:35:57 smithi062 bash[183291]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a 2024-03-24T11:35:57.643 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:35:57 smithi062 bash[183317]: Error: no container with name or ID "ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter.a" found: no such container 2024-03-24T11:35:57.947 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:35:57 smithi062 systemd[1]: Stopping Ceph alertmanager.a for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:35:57.947 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:35:57 smithi062 podman[183305]: 2024-03-24 11:35:57.643596562 +0000 UTC m=+0.603338376 container remove 79d5a02a999fc15cd0f03e2d8e2a652d6c9b8027e1077f90df4608a1ce5ec9d8 (image=quay.io/prometheus/node-exporter:v1.3.1, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-03-24T11:35:57.947 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:35:57 smithi062 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-03-24T11:35:57.947 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:35:57 smithi062 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@node-exporter.a.service: Failed with result 'exit-code'. 2024-03-24T11:35:57.947 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:35:57 smithi062 systemd[1]: Stopped Ceph node-exporter.a for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:35:57.947 INFO:journalctl@ceph.node-exporter.a.smithi062.stdout:Mar 24 11:35:57 smithi062 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@node-exporter.a.service: Consumed 4.902s CPU time. 2024-03-24T11:35:58.227 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:35:57 smithi062 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a[64072]: level=info ts=2024-03-24T11:35:57.946Z caller=main.go:557 msg="Received SIGTERM, exiting gracefully..." 2024-03-24T11:35:58.227 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:35:57 smithi062 podman[183370]: 2024-03-24 11:35:57.997682676 +0000 UTC m=+0.145222230 container died 818298d581404f0af2ff188f5c2c67ff02a44a4e5126f4f6bba0475239dbbf76 (image=quay.io/prometheus/alertmanager:v0.23.0, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-24T11:35:58.540 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:35:58 smithi062 podman[183370]: 2024-03-24 11:35:58.330375603 +0000 UTC m=+0.477915200 container cleanup 818298d581404f0af2ff188f5c2c67ff02a44a4e5126f4f6bba0475239dbbf76 (image=quay.io/prometheus/alertmanager:v0.23.0, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-24T11:35:58.540 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:35:58 smithi062 bash[183370]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a 2024-03-24T11:35:58.540 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:35:58 smithi062 podman[183385]: 2024-03-24 11:35:58.489578063 +0000 UTC m=+0.497259956 container remove 818298d581404f0af2ff188f5c2c67ff02a44a4e5126f4f6bba0475239dbbf76 (image=quay.io/prometheus/alertmanager:v0.23.0, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-24T11:35:58.540 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:35:58 smithi062 podman[183385]: 2024-03-24 11:35:58.540028163 +0000 UTC m=+0.547710027 volume remove ab49cf39fcca8ea02c4e77946c23f3ba20d60a15f62bad37adbca3364df3c102 2024-03-24T11:35:58.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:35:58 smithi062 bash[183397]: Error: no container with name or ID "ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-alertmanager.a" found: no such container 2024-03-24T11:35:58.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:35:58 smithi062 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@alertmanager.a.service: Deactivated successfully. 2024-03-24T11:35:58.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:35:58 smithi062 systemd[1]: Stopped Ceph alertmanager.a for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:35:58.977 INFO:journalctl@ceph.alertmanager.a.smithi062.stdout:Mar 24 11:35:58 smithi062 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@alertmanager.a.service: Consumed 1.406s CPU time. 2024-03-24T11:36:32.831 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-24T11:36:34.600 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:36:34 smithi139 systemd[1]: Stopping Ceph node-exporter.b for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:36:34.856 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:36:34 smithi139 podman[158781]: 2024-03-24 11:36:34.599342975 +0000 UTC m=+0.124335084 container died 8f9f4608e465db9cbd71fcc64ec2dca68e65232f12362b1e4e87818be67c6ced (image=quay.io/prometheus/node-exporter:v1.3.1, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-03-24T11:36:35.108 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:36:35 smithi139 podman[158781]: 2024-03-24 11:36:35.013741784 +0000 UTC m=+0.538733941 container cleanup 8f9f4608e465db9cbd71fcc64ec2dca68e65232f12362b1e4e87818be67c6ced (image=quay.io/prometheus/node-exporter:v1.3.1, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-03-24T11:36:35.108 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:36:35 smithi139 bash[158781]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b 2024-03-24T11:36:35.109 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:36:35 smithi139 bash[158807]: Error: no container with name or ID "ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter.b" found: no such container 2024-03-24T11:36:35.576 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:36:35 smithi139 systemd[1]: Stopping Ceph grafana.a for ebb89568-e9ce-11ee-95cd-87774f69a715... 2024-03-24T11:36:35.577 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:36:35 smithi139 podman[158795]: 2024-03-24 11:36:35.290089571 +0000 UTC m=+0.694722023 container remove 8f9f4608e465db9cbd71fcc64ec2dca68e65232f12362b1e4e87818be67c6ced (image=quay.io/prometheus/node-exporter:v1.3.1, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-03-24T11:36:35.577 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:36:35 smithi139 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-03-24T11:36:35.577 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:36:35 smithi139 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@node-exporter.b.service: Failed with result 'exit-code'. 2024-03-24T11:36:35.577 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:36:35 smithi139 systemd[1]: Stopped Ceph node-exporter.b for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:36:35.577 INFO:journalctl@ceph.node-exporter.b.smithi139.stdout:Mar 24 11:36:35 smithi139 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@node-exporter.b.service: Consumed 4.789s CPU time. 2024-03-24T11:36:35.858 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:36:35 smithi139 ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a[56945]: t=2024-03-24T11:36:35+0000 lvl=info msg="Shutdown started" logger=server reason="System signal: terminated" 2024-03-24T11:36:35.858 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:36:35 smithi139 podman[158862]: 2024-03-24 11:36:35.586687299 +0000 UTC m=+0.105803207 container died b48167e7171e96ba7723a932d5c8a043cfffda170060995294227dc1c52336b8 (image=quay.io/ceph/ceph-grafana:8.3.5, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a, vcs-type=git, maintainer=Paul Cuzner , url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-236.1648460182, io.buildah.version=1.24.2, io.openshift.expose-services=, io.openshift.tags=base rhel8, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, release=236.1648460182, build-date=2022-03-28T10:36:18.413762, com.redhat.component=ubi8-container, architecture=x86_64, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., name=ubi8, vendor=Red Hat, Inc., distribution-scope=public, version=8.5, description=Ceph Grafana Container, io.k8s.display-name=Red Hat Universal Base Image 8, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, summary=Grafana Container configured for Ceph mgr/dashboard integration) 2024-03-24T11:36:36.327 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:36:35 smithi139 podman[158862]: 2024-03-24 11:36:35.976135647 +0000 UTC m=+0.495251561 container cleanup b48167e7171e96ba7723a932d5c8a043cfffda170060995294227dc1c52336b8 (image=quay.io/ceph/ceph-grafana:8.3.5, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a, architecture=x86_64, build-date=2022-03-28T10:36:18.413762, vcs-type=git, io.buildah.version=1.24.2, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., release=236.1648460182, summary=Grafana Container configured for Ceph mgr/dashboard integration, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, version=8.5, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-236.1648460182, io.k8s.display-name=Red Hat Universal Base Image 8, io.openshift.tags=base rhel8, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, maintainer=Paul Cuzner , vendor=Red Hat, Inc., io.openshift.expose-services=, com.redhat.component=ubi8-container, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, distribution-scope=public, description=Ceph Grafana Container, name=ubi8) 2024-03-24T11:36:36.328 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:36:35 smithi139 bash[158862]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a 2024-03-24T11:36:36.328 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:36:35 smithi139 bash[158889]: Error: no container with name or ID "ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana.a" found: no such container 2024-03-24T11:36:36.608 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:36:36 smithi139 podman[158876]: 2024-03-24 11:36:36.327725141 +0000 UTC m=+0.743029360 container remove b48167e7171e96ba7723a932d5c8a043cfffda170060995294227dc1c52336b8 (image=quay.io/ceph/ceph-grafana:8.3.5, name=ceph-ebb89568-e9ce-11ee-95cd-87774f69a715-grafana-a, distribution-scope=public, release=236.1648460182, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, com.redhat.build-host=cpt-1002.osbs.prod.upshift.rdu2.redhat.com, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.k8s.display-name=Red Hat Universal Base Image 8, description=Ceph Grafana Container, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-236.1648460182, maintainer=Paul Cuzner , io.openshift.expose-services=, build-date=2022-03-28T10:36:18.413762, vendor=Red Hat, Inc., com.redhat.component=ubi8-container, io.buildah.version=1.24.2, io.openshift.tags=base rhel8, summary=Grafana Container configured for Ceph mgr/dashboard integration, version=8.5, vcs-type=git, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., name=ubi8, architecture=x86_64) 2024-03-24T11:36:36.608 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:36:36 smithi139 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@grafana.a.service: Deactivated successfully. 2024-03-24T11:36:36.608 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:36:36 smithi139 systemd[1]: Stopped Ceph grafana.a for ebb89568-e9ce-11ee-95cd-87774f69a715. 2024-03-24T11:36:36.608 INFO:journalctl@ceph.grafana.a.smithi139.stdout:Mar 24 11:36:36 smithi139 systemd[1]: ceph-ebb89568-e9ce-11ee-95cd-87774f69a715@grafana.a.service: Consumed 5.361s CPU time. 2024-03-24T11:36:59.423 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-24T11:36:59.458 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-24T11:36:59.490 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-24T11:36:59.492 DEBUG:teuthology.misc:Transferring archived files from smithi062:/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/teuthology-2024-03-22_21:08:02-orch-squid-distro-default-smithi/7617877/remote/smithi062/crash 2024-03-24T11:36:59.493 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/crash -- . 2024-03-24T11:36:59.543 INFO:teuthology.orchestra.run.smithi062.stderr:tar: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-24T11:36:59.543 INFO:teuthology.orchestra.run.smithi062.stderr:tar: Error is not recoverable: exiting now 2024-03-24T11:36:59.545 DEBUG:teuthology.misc:Transferring archived files from smithi139:/var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/teuthology-2024-03-22_21:08:02-orch-squid-distro-default-smithi/7617877/remote/smithi139/crash 2024-03-24T11:36:59.546 DEBUG:teuthology.orchestra.run.smithi139:> sudo tar c -f - -C /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/crash -- . 2024-03-24T11:36:59.646 INFO:teuthology.orchestra.run.smithi139.stderr:tar: /var/lib/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-24T11:36:59.646 INFO:teuthology.orchestra.run.smithi139.stderr:tar: Error is not recoverable: exiting now 2024-03-24T11:36:59.647 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-24T11:36:59.647 DEBUG:teuthology.orchestra.run.smithi062:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-03-24T11:36:59.685 INFO:teuthology.orchestra.run.smithi062.stdout:2024-03-24T11:12:11.895983+0000 mon.a (mon.0) 357 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-03-24T11:36:59.686 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-03-24T11:36:59.686 DEBUG:teuthology.orchestra.run.smithi062:> sudo egrep '\[SEC\]' /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-03-24T11:36:59.757 DEBUG:teuthology.orchestra.run.smithi062:> sudo egrep '\[ERR\]' /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-03-24T11:36:59.828 DEBUG:teuthology.orchestra.run.smithi062:> sudo egrep '\[WRN\]' /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-03-24T11:36:59.897 INFO:teuthology.orchestra.run.smithi062.stdout:2024-03-24T11:12:11.895983+0000 mon.a (mon.0) 357 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-03-24T11:36:59.897 INFO:tasks.cephadm:Compressing logs... 2024-03-24T11:36:59.897 DEBUG:teuthology.orchestra.run.smithi062:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-24T11:36:59.940 DEBUG:teuthology.orchestra.run.smithi139:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-24T11:36:59.964 INFO:teuthology.orchestra.run.smithi062.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-24T11:36:59.964 INFO:teuthology.orchestra.run.smithi062.stderr:: No such file or directory 2024-03-24T11:36:59.964 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-mon.a.log 2024-03-24T11:36:59.964 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph.audit.log 2024-03-24T11:36:59.964 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-client.rgw.foo.smithi062.sacpds.log 2024-03-24T11:36:59.967 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-client.rgw.smpl.smithi062.xvpxnk.log 2024-03-24T11:36:59.967 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-client.rgw.foo.smithi062.sacpds.log: gzip -5 --verbose -- /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-osd.1.log 2024-03-24T11:36:59.967 INFO:teuthology.orchestra.run.smithi062.stderr: 76.9% -- replaced with /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-client.rgw.foo.smithi062.sacpds.log.gz 2024-03-24T11:36:59.967 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-client.rgw.smpl.smithi062.xvpxnk.log: gzip -5 --verbose -- /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-osd.3.log 2024-03-24T11:36:59.967 INFO:teuthology.orchestra.run.smithi062.stderr: 76.8% -- replaced with /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-client.rgw.smpl.smithi062.xvpxnk.log.gz 2024-03-24T11:36:59.967 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-mgr.y.log 2024-03-24T11:36:59.967 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/tcmu-runner.log 2024-03-24T11:36:59.967 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-osd.0.log 2024-03-24T11:36:59.967 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/tcmu-runner.log: 87.1% -- replaced with /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/tcmu-runner.log.gz 2024-03-24T11:36:59.967 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph.cephadm.log 2024-03-24T11:36:59.968 INFO:teuthology.orchestra.run.smithi139.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-24T11:36:59.971 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-24T11:36:59.971 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-osd.6.log 2024-03-24T11:36:59.971 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-client.rgw.smpl.smithi139.pbtrmp.log 2024-03-24T11:36:59.971 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph.audit.log 2024-03-24T11:36:59.975 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose --/var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-client.rgw.smpl.smithi139.pbtrmp.log: /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-osd.4.log 2024-03-24T11:36:59.975 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph.audit.log: 76.9% -- replaced with /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-client.rgw.smpl.smithi139.pbtrmp.log.gzgzip -5 --verbose -- /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-client.rgw.foo.smithi139.yljesh.log 2024-03-24T11:36:59.975 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-03-24T11:36:59.975 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-osd.7.log 2024-03-24T11:36:59.975 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-osd.4.log: /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-client.rgw.foo.smithi139.yljesh.log: 76.8% -- replaced with /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-client.rgw.foo.smithi139.yljesh.log.gz 2024-03-24T11:36:59.975 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-osd.5.log 2024-03-24T11:36:59.980 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph.log 2024-03-24T11:36:59.983 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-mgr.x.log 2024-03-24T11:36:59.987 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-osd.2.log 2024-03-24T11:36:59.991 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph.cephadm.log 2024-03-24T11:36:59.992 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-mgr.x.log: 93.1% -- replaced with /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph.audit.log.gz 2024-03-24T11:36:59.994 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-volume.log 2024-03-24T11:36:59.995 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-osd.2.log: 95.0% -- replaced with /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph.cephadm.log.gz 2024-03-24T11:36:59.998 INFO:teuthology.orchestra.run.smithi139.stderr: 89.6% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-24T11:36:59.999 INFO:teuthology.orchestra.run.smithi062.stderr: 93.3% 90.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-24T11:36:59.999 INFO:teuthology.orchestra.run.smithi062.stderr: -- replaced with /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph.log.gz 2024-03-24T11:37:00.000 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-mon.c.log 2024-03-24T11:37:00.006 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph.log 2024-03-24T11:37:00.009 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-volume.log: 95.6% -- replaced with /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph.audit.log.gz 2024-03-24T11:37:00.010 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph.cephadm.log: 91.5% -- replaced with /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph.cephadm.log.gz 2024-03-24T11:37:00.011 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-volume.log 2024-03-24T11:37:00.016 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph.log: 88.2% -- replaced with /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph.log.gz 2024-03-24T11:37:00.016 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-mon.b.log 2024-03-24T11:37:00.067 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-volume.log: /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-mon.b.log: 88.7% -- replaced with /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-mgr.x.log.gz 2024-03-24T11:37:00.067 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-mon.c.log: 92.4% -- replaced with /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-volume.log.gz 2024-03-24T11:37:00.079 INFO:teuthology.orchestra.run.smithi139.stderr: 92.5% -- replaced with /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-volume.log.gz 2024-03-24T11:37:00.572 INFO:teuthology.orchestra.run.smithi139.stderr: 92.8% -- replaced with /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-mon.b.log.gz 2024-03-24T11:37:00.936 INFO:teuthology.orchestra.run.smithi062.stderr: 92.6% -- replaced with /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-mon.c.log.gz 2024-03-24T11:37:01.285 INFO:teuthology.orchestra.run.smithi139.stderr: 93.3% -- replaced with /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-osd.6.log.gz 2024-03-24T11:37:01.542 INFO:teuthology.orchestra.run.smithi062.stderr: 93.5% -- replaced with /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-osd.2.log.gz 2024-03-24T11:37:01.630 INFO:teuthology.orchestra.run.smithi062.stderr: 88.6% -- replaced with /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-mgr.y.log.gz 2024-03-24T11:37:01.737 INFO:teuthology.orchestra.run.smithi139.stderr: 93.6% -- replaced with /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-osd.7.log.gz 2024-03-24T11:37:01.874 INFO:teuthology.orchestra.run.smithi139.stderr: 93.5% -- replaced with /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-osd.5.log.gz 2024-03-24T11:37:01.876 INFO:teuthology.orchestra.run.smithi062.stderr: 93.7% -- replaced with /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-osd.0.log.gz 2024-03-24T11:37:02.083 INFO:teuthology.orchestra.run.smithi139.stderr: 93.7% -- replaced with /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-osd.4.log.gz 2024-03-24T11:37:02.084 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-03-24T11:37:02.085 INFO:teuthology.orchestra.run.smithi139.stderr:real 0m2.130s 2024-03-24T11:37:02.085 INFO:teuthology.orchestra.run.smithi139.stderr:user 0m7.481s 2024-03-24T11:37:02.085 INFO:teuthology.orchestra.run.smithi139.stderr:sys 0m0.339s 2024-03-24T11:37:02.219 INFO:teuthology.orchestra.run.smithi062.stderr: 93.7% -- replaced with /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-osd.1.log.gz 2024-03-24T11:37:02.453 INFO:teuthology.orchestra.run.smithi062.stderr: 91.9% -- replaced with /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-mon.a.log.gz 2024-03-24T11:37:02.461 INFO:teuthology.orchestra.run.smithi062.stderr: 93.6% -- replaced with /var/log/ceph/ebb89568-e9ce-11ee-95cd-87774f69a715/ceph-osd.3.log.gz 2024-03-24T11:37:02.463 INFO:teuthology.orchestra.run.smithi062.stderr: 2024-03-24T11:37:02.463 INFO:teuthology.orchestra.run.smithi062.stderr:real 0m2.512s 2024-03-24T11:37:02.463 INFO:teuthology.orchestra.run.smithi062.stderr:user 0m12.908s 2024-03-24T11:37:02.463 INFO:teuthology.orchestra.run.smithi062.stderr:sys 0m0.440s 2024-03-24T11:37:02.463 INFO:tasks.cephadm:Archiving logs... 2024-03-24T11:37:02.464 DEBUG:teuthology.misc:Transferring archived files from smithi062:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-03-22_21:08:02-orch-squid-distro-default-smithi/7617877/remote/smithi062/log 2024-03-24T11:37:02.465 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-24T11:37:04.057 DEBUG:teuthology.misc:Transferring archived files from smithi139:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-03-22_21:08:02-orch-squid-distro-default-smithi/7617877/remote/smithi139/log 2024-03-24T11:37:04.058 DEBUG:teuthology.orchestra.run.smithi139:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-24T11:37:05.133 INFO:tasks.cephadm:Removing cluster... 2024-03-24T11:37:05.133 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 --force 2024-03-24T11:37:05.431 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ebb89568-e9ce-11ee-95cd-87774f69a715 --force 2024-03-24T11:37:05.722 INFO:tasks.cephadm:Removing cephadm ... 2024-03-24T11:37:05.722 DEBUG:teuthology.orchestra.run.smithi062:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-24T11:37:05.738 DEBUG:teuthology.orchestra.run.smithi139:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-24T11:37:05.752 INFO:tasks.cephadm:Teardown complete 2024-03-24T11:37:05.752 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-24T11:37:05.763 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-24T11:37:05.763 DEBUG:teuthology.orchestra.run.smithi062:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-24T11:37:05.780 DEBUG:teuthology.orchestra.run.smithi139:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-24T11:37:05.792 INFO:teuthology.orchestra.run.smithi062.stderr:bash: line 1: ntpq: command not found 2024-03-24T11:37:05.796 INFO:teuthology.orchestra.run.smithi062.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-24T11:37:05.796 INFO:teuthology.orchestra.run.smithi062.stdout:=============================================================================== 2024-03-24T11:37:05.796 INFO:teuthology.orchestra.run.smithi062.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 61 +863us[ +874us] +/- 95ms 2024-03-24T11:37:05.796 INFO:teuthology.orchestra.run.smithi062.stdout:^* hv02.front.sepia.ceph.com 3 6 377 58 +67us[ +79us] +/- 62ms 2024-03-24T11:37:05.796 INFO:teuthology.orchestra.run.smithi062.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 59 -676us[ -665us] +/- 70ms 2024-03-24T11:37:05.797 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T11:37:05.806 INFO:teuthology.orchestra.run.smithi139.stderr:bash: line 1: ntpq: command not found 2024-03-24T11:37:05.810 INFO:teuthology.orchestra.run.smithi139.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-24T11:37:05.810 INFO:teuthology.orchestra.run.smithi139.stdout:=============================================================================== 2024-03-24T11:37:05.810 INFO:teuthology.orchestra.run.smithi139.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 59 +839us[ +827us] +/- 95ms 2024-03-24T11:37:05.810 INFO:teuthology.orchestra.run.smithi139.stdout:^* hv02.front.sepia.ceph.com 3 8 377 57 +76us[ +64us] +/- 62ms 2024-03-24T11:37:05.810 INFO:teuthology.orchestra.run.smithi139.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 59 -697us[ -709us] +/- 70ms 2024-03-24T11:37:05.810 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T11:37:05.811 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-24T11:37:05.821 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-24T11:37:05.822 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-24T11:37:05.844 DEBUG:teuthology.orchestra.run.smithi062:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-24T11:37:05.961 DEBUG:teuthology.orchestra.run.smithi139:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-24T11:37:06.068 DEBUG:teuthology.orchestra.run.smithi062:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-24T11:37:06.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-24T11:37:06.101 DEBUG:teuthology.orchestra.run.smithi139:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-24T11:37:06.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-24T11:37:06.134 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-24T11:37:06.144 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-24T11:37:06.163 INFO:teuthology.task.internal:Duration was 1940.890443 seconds 2024-03-24T11:37:06.163 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-24T11:37:06.194 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-24T11:37:06.195 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-24T11:37:06.197 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-24T11:37:06.238 INFO:teuthology.orchestra.run.smithi062.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-24T11:37:06.240 INFO:teuthology.orchestra.run.smithi139.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-24T11:37:06.515 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-24T11:37:06.516 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi062.front.sepia.ceph.com 2024-03-24T11:37:06.516 DEBUG:teuthology.orchestra.run.smithi062:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-24T11:37:06.575 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi139.front.sepia.ceph.com 2024-03-24T11:37:06.575 DEBUG:teuthology.orchestra.run.smithi139:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-24T11:37:06.602 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-24T11:37:06.602 DEBUG:teuthology.orchestra.run.smithi062:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-24T11:37:06.617 DEBUG:teuthology.orchestra.run.smithi139:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-24T11:37:06.671 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-24T11:37:06.671 DEBUG:teuthology.orchestra.run.smithi062:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-24T11:37:06.685 DEBUG:teuthology.orchestra.run.smithi139:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-24T11:37:07.479 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-24T11:37:07.489 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-24T11:37:07.489 DEBUG:teuthology.orchestra.run.smithi062:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-24T11:37:07.515 DEBUG:teuthology.orchestra.run.smithi139:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-24T11:37:07.542 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-24T11:37:07.552 DEBUG:teuthology.orchestra.run.smithi062:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-24T11:37:07.557 DEBUG:teuthology.orchestra.run.smithi139:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-24T11:37:07.580 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern = core 2024-03-24T11:37:07.608 INFO:teuthology.orchestra.run.smithi139.stdout:kernel.core_pattern = core 2024-03-24T11:37:07.627 DEBUG:teuthology.orchestra.run.smithi062:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-24T11:37:07.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-24T11:37:07.656 DEBUG:teuthology.orchestra.run.smithi139:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-24T11:37:07.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-24T11:37:07.682 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-24T11:37:07.692 INFO:teuthology.task.internal:Transferring archived files... 2024-03-24T11:37:07.693 DEBUG:teuthology.misc:Transferring archived files from smithi062:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-03-22_21:08:02-orch-squid-distro-default-smithi/7617877/remote/smithi062 2024-03-24T11:37:07.693 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-24T11:37:07.755 DEBUG:teuthology.misc:Transferring archived files from smithi139:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-03-22_21:08:02-orch-squid-distro-default-smithi/7617877/remote/smithi139 2024-03-24T11:37:07.755 DEBUG:teuthology.orchestra.run.smithi139:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-24T11:37:07.805 INFO:teuthology.task.internal:Removing archive directory... 2024-03-24T11:37:07.805 DEBUG:teuthology.orchestra.run.smithi062:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-24T11:37:07.807 DEBUG:teuthology.orchestra.run.smithi139:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-24T11:37:07.849 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-24T11:37:07.860 INFO:teuthology.task.internal:Not uploading archives. 2024-03-24T11:37:07.860 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-24T11:37:07.886 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-24T11:37:07.886 DEBUG:teuthology.orchestra.run.smithi062:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-24T11:37:07.889 DEBUG:teuthology.orchestra.run.smithi139:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-24T11:37:07.900 INFO:teuthology.orchestra.run.smithi062.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 24 11:37 /home/ubuntu/cephtest 2024-03-24T11:37:07.903 INFO:teuthology.orchestra.run.smithi139.stdout: 265401 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 24 11:37 /home/ubuntu/cephtest 2024-03-24T11:37:07.904 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-24T11:37:07.920 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-24T11:37:07.987 INFO:teuthology.nuke:Checking targets against current locks 2024-03-24T11:37:08.011 DEBUG:teuthology.nuke:shortname: smithi062 2024-03-24T11:37:08.011 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-24T11:37:08.033 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi062.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-22_21:08:02-orch-squid-distro-default-smithi/7617877', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-24 10:56:03.221993', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMKVmzq25YeKw1MgfUfOJywdEoM1CYUWDFwQCCeWPaoDie4Uu9xnlKszomsaVLFYHsRNTz8HZS2SyP3g+hl99BU='} 2024-03-24T11:37:08.036 DEBUG:teuthology.nuke:shortname: smithi139 2024-03-24T11:37:08.036 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-24T11:37:08.057 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi139.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-22_21:08:02-orch-squid-distro-default-smithi/7617877', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-24 10:56:03.220094', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJm+byHDUb5PVB6DkMlCOZKTd40GZgfT9nsEGI03HSonCLLa90OYUB+TTEaRVDHiNoRyyHJRDpghgtY2xrRgWpw='} 2024-03-24T11:37:08.085 INFO:teuthology.orchestra.console.smithi062:Power off 2024-03-24T11:37:08.085 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-24T11:37:08.108 DEBUG:teuthology.orchestra.console.smithi062:power off output: Chassis Power Control: Down/Off 2024-03-24T11:37:08.109 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-24T11:37:08.123 INFO:teuthology.orchestra.console.smithi139:Power off 2024-03-24T11:37:08.123 DEBUG:teuthology.orchestra.console.smithi139:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-24T11:37:08.132 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is on 2024-03-24T11:37:08.146 DEBUG:teuthology.orchestra.console.smithi139:power off output: Chassis Power Control: Down/Off 2024-03-24T11:37:08.146 DEBUG:teuthology.orchestra.console.smithi139:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-24T11:37:08.169 DEBUG:teuthology.orchestra.console.smithi139:check power output: Chassis Power is on 2024-03-24T11:37:12.134 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-24T11:37:12.170 DEBUG:teuthology.orchestra.console.smithi139:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-24T11:37:12.245 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is on 2024-03-24T11:37:12.278 DEBUG:teuthology.orchestra.console.smithi139:check power output: Chassis Power is on 2024-03-24T11:37:16.250 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-24T11:37:16.278 DEBUG:teuthology.orchestra.console.smithi139:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-24T11:37:16.360 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is on 2024-03-24T11:37:16.389 DEBUG:teuthology.orchestra.console.smithi139:check power output: Chassis Power is off 2024-03-24T11:37:16.490 INFO:teuthology.orchestra.console.smithi139:Power off completed 2024-03-24T11:37:20.362 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-24T11:37:20.471 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is off 2024-03-24T11:37:20.572 INFO:teuthology.orchestra.console.smithi062:Power off completed 2024-03-24T11:37:20.676 INFO:teuthology.run:Summary data: description: orch/cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/connectivity} duration: 1940.8904428482056 failure_reason: '"2024-03-24T11:12:11.895983+0000 mon.a (mon.0) 357 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log' owner: scheduled_teuthology@teuthology success: false 2024-03-24T11:37:20.676 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-24T11:37:20.769 INFO:teuthology.run:FAIL